{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Loss Functions\n",
    "\n",
    "When fitting data to labels we need to measure the degree of goodness of fit. This sounds obvious but isn't quite so straightforward. In fact, there are entire fields of statistics that focus solely on that (e.g. robust statistics). In this notebook we'll discuss a number of ways how we can measure whether our model is doing well. As a side-benefit, we'll get to know the loss function layers in ``gluon``. We begin with our default import ritual."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import mxnet as mx\n",
    "import mxnet.gluon as gluon\n",
    "from mxnet import nd, autograd\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import mxnet.autograd as ag\n",
    "import math\n",
    "mx.random.seed(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regression \n",
    "\n",
    "## L1 loss\n",
    "\n",
    "As we discussed in the introduction, regression describes the cases where we want to estimate some real valued number $f(x) \\in \\mathbb{R}$ to match an observation $y$. A natural idea of measuring the distance would be to compute $|y - f(x)|$. This makes sense, e.g. if we need to estimate how much it might cost to manufacture a product: if we estimate too low, we will incur a loss due to underestimation. If we overprice it, we will sell fewer products (here we're making the unrealistic assumption that both are equally bad). In math, the loss function is\n",
    "\n",
    "$$l(y,f) = |y-f|$$\n",
    "\n",
    "Let's compute it with ``gluon`` and also its gradient. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8VfX9x/HXN8nNJAsIM2xkZpKFpYIDEQcIKCojJCGI\nWmtV6qY/3K1WW61KVSQkIYDiwlEQK4oClWYS9t5hJpBJErK+vz8gqa0gJLn3njs+z8cjDwm595z3\nheubc8/4HKW1RgghhONwMTqAEEII85JiF0IIByPFLoQQDkaKXQghHIwUuxBCOBgpdiGEcDBS7EII\n4WCk2IUQwsFIsQshhINxM2Kl7du31z179jRi1UIIYbdyc3OLtNZBl3qcIcXes2dPcnJyjFi1EELY\nLaXUwct5nOyKEUIIByPFLoQQDkaKXQghHIwUuxBCOBgpdiGEcDBmOStGKXUAKAfqgTqtdbQ5liuE\nEKL5zHm64zVa6yIzLk8IIUQL2NWumPzDJbzzw16jYwghRItk7juFNW5Haq5i18A/lVK5SqmZF3qA\nUmqmUipHKZVTWFjYopV8mlfAS1/tIPVf+1uTVQghrC71X/u5c96/Wb75mMXXZa5dMcO01keVUh2A\nb5RSO7TWa376AK31PGAeQHR0dIv+yZpzyyCOl1bz7JfbCPA2MT4yuPXJhRDCwpZtKODZL7cxalBH\nRg/uZPH1mWWLXWt99Px/TwLLgFhzLPd/ubm68MakSH7Vpx2PfLSJb7efsMRqhBDCbL7bcYJHPtrE\nr/q0441Jkbi5Wn4PeKvXoJTyUUr5Nv4aGAVsae1yL8bT5Mq8adEM7uLHbxbnkbnvlKVWJYQQrZK1\n/zT3LcpjcBc/5k2LxtPkapX1muOfjo7AOqXURiALWK61XmmG5V5UGw83UhNj6BroxYz0HLYeLbXk\n6oQQotm2Hi0lOS2broFepCXF0sbDejMXW13sWut9Wuvw81+DtdYvmiPYpbRr48Gi5Dh8Pd1IWJDF\n/qIz1litEEJc0v6iMyQsyMLX041FyXG09XG36vrt6nTH/9UlwIuMGXE0aJg6P5PjpdVGRxJCOLnj\npdVMnZ9Jg4aMGXF0CfCyega7LnaAPkFtSE+KpaSyhviUTIrP1BgdSQjhpEoqa5i2IJPSqlrSk2Lp\nE9TGkBx2X+wAocH+vJcQzcHTlSSlZXPmbJ3RkYQQTubM2ToSU7M5cKqS96ZFExrsb1gWhyh2gF/1\nac+bkyLZVFDCvYtyOVtXb3QkIYSTOFtXz72LctlUUMKbkyK5sk87Q/M4TLED3DC4Ey/fFsba3UXM\nWrqR+gbLX7orhHBu9Q2aWUs3snZ3ES/fFsYNVrgA6VIMueepJU2M7kZpVS0vLN+On5cbfxwfilLK\n6FhCCAekteYPn21h+eZj/OHmgUyM7mZ0JMABix1gxlW9OX2mhr9/v5dAb3ceGz3A6EhCCAf0ytc7\neT/rEPdf04cZV/U2Ok4Thyx2gEdv6E9JVW1Tud893Hb+0IUQ9u+9Nfv4+/d7mRzXnUdG9Tc6zn9x\n2GJXSvH8rSGUVtXy4ort+HubuMNGPiYJIezbhzmHeXHFdm4O68zzt4bY3O5ehy12AFcXxWt3RFBW\nVcsTn2zCz9PE6BDjD2wIIezX11uP88Qnm7jqiva8dkcEri62VergYGfFXIi7mwvvTI0ivFsAv3t/\nAz/ulZs8CSFa5se9RTywZAPh3QJ4Z2oU7m62WaG2mcrMfM4PDevZ3pu703PYVFBidCQhhJ3ZVFDC\n3ek59GzvTWpiDD5WHOrVXE5R7AAB3u4snB5HoI87ianZ7DlZYXQkIYSd2HOygsTUbAJ93MlIjiPA\n27pDvZrLaYodoJO/JxnJcbgoiE/J5EhJldGRhBA27khJFdNSMnFRsCg5jo5+nkZHuiSnKnaAXu19\nSJ8eS0V1HfEpmZyqOGt0JCGEjTpVcZb4lEzKq+tInx5Lz/Y+Rke6LE5X7ACDu/iTkhjDkeIqElOz\nKa+uNTqSEMLGVJwf6nWkuIqUxBgGdzFuqFdzOWWxA8T2asvbU4ew7VgZMxfmUl0rQ8OEEOdU19Zz\nd3oO24+V8fbUIcT2amt0pGZx2mIHuHZAR/4yMZz1+07xwPsbqKtvMDqSEMJgdfUN/O79Dazfd4pX\nJ4Zz7YCORkdqNqcudoBxkV15Zswgvtl2gic+3YzWMhFSCGeltebJTzfzz20neGbMIMZFdjU6UovY\n7omYVpQ4rBfFlbX87dvdBHqbeOqmgTZ3ibAQwrK01vxxxXY+yi3gweuuIHFYL6MjtZgU+3kPjbyC\nksoa3lu7nwBvd+6/pq/RkYQQVvT2D3t5b+1+Eq7swUMjrzA6TqtIsZ+nlOLpMYMpqarlla93Eujt\nzuS47kbHEkJYwZLMQ/x55U5ujejC02MG2/0ndin2n3BxUbw6MZyyqlpmf7YZfy8TN4d1NjqWEMKC\nVmw+xuzPNnNN/yBenRiOiw0O9Woupz94+r9Mri78fUoU0T0CeWjpBtbsKjQ6khDCQtbuLuTBDzYQ\n3SOQv0+JwuTqGJVotlehlHJVSm1QSv3DXMs0ipe7K/MTYugT1IZ7MnLJO1RsdCQhhJltOFTMPRm5\n9Alqw/yEGLzcXY2OZDbm/OfpQWC7GZdnKH8vEwuTY+ng50FSaja7TpQbHUkIYSa7TpSTmJpNkK8H\nC5Nj8fcyGR3JrMxS7EqpYOBmYL45lmcrOvh6sig5Dg83F+JTMjl8utLoSEKIVjp8upL4lEw83FxY\nlBxHB1/bH+rVXObaYn8deAy46KWbSqmZSqkcpVROYaH97Lfu1tabjOQ4qmsbiE/JpLBchoYJYa8K\ny88N9aqubSAjOY5ubb2NjmQRrS52pdQtwEmtde4vPU5rPU9rHa21jg4KCmrtaq2qfydfFiTGcKLs\nLNMWZFFaJUPDhLA3ZdW1JCzI4kTZWRYkxtC/k6/RkSzGHFvsw4CxSqkDwAfAtUqpRWZYrk2J6hHI\nO/FR7DlZzoz0bKpqZGiYEPaiqqaeGWk57D5ZzjvxUUT1CDQ6kkW1uti11k9qrYO11j2Bu4DvtNZT\nW53MBo3oF8Rf74gg52Ax9y/Jo1aGhglh82rrG/jtkjyyD57mr3dEMKKffe0xaAnHOGnTisaEd+GF\ncSF8t+Mkj360kYYGGRomhK1qaNA89vEmvt1xkhfGhTAmvIvRkazCrFeeaq2/B7435zJt0ZS4HpRU\nnhs9EODtztNjBtn9JchCOBqtNc/9YxvLNhzh0Rv6MyWuh9GRrEZGCrTQb67uw+kzNaSs20+gtzsP\n2vnQICEczRvf7iHtxwMk/7oXv7m6j9FxrEqKvYWUUsy+aSAllbW8tmoXAd4mEn7V0+hYQghg4foD\nvLZqF7cNCWa2E47hlmJvBRcXxcu3hVJWXcvTX2wlwNvErRH2OZhfCEfxef4Rnv5iK9cP6sjLt4U6\nxFCv5pKDp63k5urCm5MiievVlt9/uJHVO04aHUkIp7V6x0l+/+FGYnu25c1Jkbg5yFCv5nLOV21m\nniZX5idEM6CzL/cuyiX7wGmjIwnhdLIPnOa+xbkM6OzL/IRoPE2OM9SruaTYzcTX00RaUixdA7yY\nnpbNtqNlRkcSwmlsP1bG9LRsuvh7kZYUi6+nYw31ai4pdjNq38aDjBlxtPFwY9qCLA4UnTE6khAO\n7+CpM8SnZNHGw42MGXG0b+NhdCTDSbGbWdcALzKSY6lvaGBqSiYnyqqNjiSEwzpRVs3UlEzqGxrI\nSD73iVlIsVtE3w6+pCXFUnymhmkpWZRU1hgdSQiHU1pZy7SULE5X1JCWFEvfDo471Ku5pNgtJLxb\nAPOmRbO/6AzT07KprKkzOpIQDqOypo6ktCz2F51h3rRowrsFGB3JpkixW9Cwvu15Y1IE+YdLuCcj\nl5o6GRomRGvV1DVw76I88g+X8MakCIb1bW90JJsjxW5ho0M686cJoazdXcSsD/Opl6FhQrRYfYNm\n1of5rNlVyJ8mhDI6pLPRkWySXHlqBXfGdKekspY/fbUDfy8TL4wLcbpLnIVoLa01cz7fwj82HePJ\nGwdwZ0x3oyPZLCl2K7lnRB+KK2t554e9BHq788gN/Y2OJIRd+es3u1iceYh7R/ThnhHONdSruaTY\nrejx0f0pqazhrdV7CPA2MeOq3kZHEsIupKzbz5vf7eGumG48Plo2ii5Fit2KlFK8OD6U0qpaXli+\nnQBvd26PCjY6lhA27ZPcAp7/xzZuDOnEi+NDZTfmZZCDp1bm6qJ4/a4Ift23PY9/solvtp0wOpIQ\nNuubbSd47JNN/Lpve16/KwJXJ5zU2BJS7AbwcHPl3fgoQrr6c/+SPNbvPWV0JCFszr/3neL+JXmE\ndPXn3fgoPNycd6hXc0mxG8THw420xBi6t/Xm7oU5bC4oNTqSEDZjy5FSZqTn0L2tN2mJMfh4yF7j\n5pBiN1CgjzsZybH4e5lISM1ib2GF0ZGEMNy+wgoSFmTh72UiIzmWQB93oyPZHSl2g3X2Pzc0TAHT\nUrI4WlJldCQhDHOstIr4lCwAMpJj6ewvQ71aQordBvQOakP69FjKqmqJT8nk9BkZGiacz+kzNcSn\nZFFWVUv69Fh6B7UxOpLdkmK3ESFd/ZmfEE1BcRWJqVlUnJWhYcJ5VJytIyk1i8OnK3kvIZqQrv5G\nR7JrrS52pZSnUipLKbVRKbVVKfWsOYI5o7je7Zg7eQhbj5Yxc2EOZ+vqjY4khMWdravnnowcthwt\nY+7kIQzt3c7oSHbPHFvsZ4FrtdbhQAQwWik11AzLdUojB3XkldvD+HHvKR58P5+6epkIKRxXfYPm\nwffz+deeU7xyexgjB3U0OpJDaHWx63MaT+cwnf+SEYatMGFIMHNuGcTKrceZvWwLWssfp3A8Wmue\n+nQzK7ceZ84tg5gwRK7CNheznByqlHIFcoG+wFytdaY5luvMpv+6FyWVNbzx3bm5Mk/eNNDoSEKY\n1Usrd7A05zAPXNuX6b/uZXQch2KWYtda1wMRSqkAYJlSKkRrveWnj1FKzQRmAnTvLuM2L8fD1/ej\nuLKWd9fsI9DHnXtlop1wEO/8sJd3f9hH/NAezLq+n9FxHI5Zz4rRWpcA3wOjL/CzeVrraK11dFBQ\nkDlX67CUUjw7djBjwrvw0lc7+CDrkNGRhGi1pdmHeOmrHYwJ78KzYwfLUC8LMMdZMUHnt9RRSnkB\nI4EdrV2uOMfFRfGXieGM6BfEU8s289XmY0ZHEqLFVm45xpOfbmZEvyD+MjEcFxnqZRHm2GLvDKxW\nSm0CsoFvtNb/MMNyxXnubi68MzWKyO6BPPhBPut2FxkdSYhmW7e7iN+9n09k90DenjoEdze5jMZS\nzHFWzCatdaTWOkxrHaK1fs4cwcR/83J3ZUFCDL2DfJiZkUP+4RKjIwlx2fIPlzAzI4feQT4sSIjB\n212GelmS/JNpR/y9TSycHkv7Nh4kpmax+0S50ZGEuKQ9J8tJSs2ifRsPFk6Pxd/bZHQkhyfFbmc6\n+HmyKDkOk6sL8SlZFBRXGh1JiIsqKK5k6vws3FxdWJQcRwc/T6MjOQUpdjvUvZ03C6fHUllTR3xK\nFkUVZ42OJMTPFFWcJT4li8qaOhZOj6V7O2+jIzkNKXY7NbCzHwsSYzhWWkXCgizKqmuNjiREk/Lq\nWhIWZHGstIoFiTEM7OxndCSnIsVux6J7tuWdqVHsPF7OjPQcqmtlaJgwXnVtPTPSc9h5vJy3p0YR\n3bOt0ZGcjhS7nbu6fwf+emcE2QdO89sledTK0DBhoLr6Bn67ZANZB07zlzvCuaZ/B6MjOSUpdgcw\nNrwLz90awqrtJ3n84000NMjQMGF9DQ2axz7ZxKrtJ3hu7GBujehqdCSnJSeTOoj4oT0oPlPDX7/Z\nRYC3O/93y0C5VFtYjdaaF5Zv59O8I8y6vh/xV/Y0OpJTk2J3IA9c25fiyhoW/Gs/gd4mHrjuCqMj\nCScxd/UeFvxrP0nDevLAtX2NjuP0pNgdiFKK/7t5EKWVtfzlm10E+LgTP7SH0bGEg8v490Fe/ecu\nxkd25f9uHiSfFG2AFLuDcXFRvHx7GGXVtcz5fAv+XibGhncxOpZwUF9sPMqcz7cwcmAH/nx7mAz1\nshFy8NQBmVxdeGvyEGJ6tmXW0ny+33nS6EjCAX2/8ySzluYT07Mtb00egslV6sRWyN+Eg/I0uTI/\nIZr+nXy5d1EuuQdPGx1JOJDcg6e5b1Ee/Tr6Mj8hGk+Tq9GRxE9IsTswP08T6dNj6ezvRVJqNtuP\nlRkdSTiAHcfLSErNpqOfB+nTY/HzlKFetkaK3cE1TtTzdndj2oIsDp2SoWGi5Q6dqmRaShbe7m5k\nJMcR5OthdCRxAVLsTqBbW28ykmOprW9gakomJ8uqjY4k7NDJ8mriF2RSU99ARnIs3drKUC9bJcXu\nJK7o6EtaUixFFWeZtiCL0koZGiYuX2lVLdNSsigsP0tqYgxXdPQ1OpL4BVLsTiSiWwDz4qPZW1jB\n9PRsKmvqjI4k7EBVTT3JadnsLazg3fhzt2gUtk2K3cn8+or2/O2uSDYcKua+RXnU1MnQMHFxtfUN\n3Lc4l7xDxfztrkiuuiLI6EjiMkixO6GbQjvz4vhQfthVyCMfbZShYeKCGho0j3y0ke93FvLi+FBu\nCu1sdCRxmeTKUyc1KbY7JZW1vLxyB/5eJp67dbBcCi6aaK155sutfJ5/lMdG92dSbHejI4lmkGJ3\nYvdd3YeSyhreXbOPQB93Zl3fz+hIwka8tmo3C9cfZObw3tw3oo/RcUQzSbE7uSduHEBxZQ1vfLub\nQG8TScN6GR1JGCz1X/t549vd3BEdzJM3DpBPcnZIit3JKaX44/hQSqtqefbLbfh7mZgwJNjoWMIg\nyzYU8OyX27hhcEf+OD5USt1OtfrgqVKqm1JqtVJqu1Jqq1LqQXMEE9bj5urC3+6K5Fd92vHox5tY\nte2E0ZGEAb7dfoJHPtrEr/q04293ReImQ73sljn+5uqA32utBwJDgfuVUoPMsFxhRZ4mV+ZNi2Zw\nFz/uX5JH5r5TRkcSVpS57xS/WZzH4C5+zJsmQ73sXauLXWt9TGudd/7X5cB2QG52aIfaeLiRlhRL\ncKAXM9Jz2HKk1OhIwgq2Hi1lRnoOwYFepCXF0sZD9tDaO7N+1lJK9QQigUxzLldYT1sfdzKS4/D1\ndCMxNYv9RWeMjiQsaH/RGRIWZOHreW6oV1sfd6MjCTMwW7ErpdoAnwAPaa1/Nh9WKTVTKZWjlMop\nLCw012qFBXQJ8CJjRhxaw9T5mRwrrTI6krCA46XVTJ2fSYOGjBlxdAnwMjqSMBOzFLtSysS5Ul+s\ntf70Qo/RWs/TWkdrraODguSyZFvXJ6gNaUmxTcOfis/UGB1JmFHxmRriUzIpraolPSmWPkFtjI4k\nzMgcZ8UoIAXYrrX+a+sjCVsRGuzPe9OiOXi6kqS0bM6claFhjuDM2TqS0rI5eLqS96ZFExrsb3Qk\nYWbm2GIfBsQD1yql8s9/3WSG5QobcGWfdrw1KZLNR0q5JyOXs3X1RkcSrXC2rp57F+WyqaCEtyZF\ncmWfdkZHEhZgjrNi1mmtldY6TGsdcf5rhTnCCdswanAnXr4tjHV7inh4aT71MjTMLtU3aGYt3cja\n3UW8fFsYowZ3MjqSsBA5r0lcltujgimprOGF5dvx99osVyXaGa01f/hsC8s3H+MPNw9kYnQ3oyMJ\nC5JiF5dtxlW9Ka6sYe7qvQR4u/P46AFGRxKX6ZWvd/J+1iHuv6YPM67qbXQcYWFS7KJZHhnVn+LK\nWt7+fi+B3iZmDpfJf7buvTX7+Pv3e5kc151HRvU3Oo6wAil20SxKKZ6/NYTSqlr+uGIHAV7u3BEj\nH+tt1Yc5h3lxxXZuDuvM87eGyO4zJyHFLprN1UXx2h0RlFXV8sSnm/DzMjE6RA7E2Zqvtx7niU82\ncdUV7XntjghcXaTUnYWMbxMt4u7mwrvxUYR3C+B372/gxz1FRkcSP/Hj3iIeWLKB8G4BvBsfhbub\n/K/uTORvW7SYt7sbqYkx9Grvw90Lc9h4uMToSALYVFDC3ek59GzvTWpiDN7u8sHc2Uixi1YJ8HZn\nYXIsgT7uJKZmsedkudGRnNqekxUkpmYTeH6YW4C3DPVyRlLsotU6+nmyKDkOVxcX4lOyOFIiQ8OM\ncKSkiviUTFyUYlFyHB39PI2OJAwixS7Momd7HxZOj6XibB3xKZmcqjhrdCSncqriLPEpmVScrWPh\n9Fh6tvcxOpIwkBS7MJtBXfxYkBjD0ZIqElKzKK+uNTqSUyivriUxNZsjxVWkJMQwqIuf0ZGEwaTY\nhVnF9GzL21Oi2HGsnLsX5lBdK0PDLKm6tp6ZC3PZfqyMt6cOIbZXW6MjCRsgxS7M7poBHXh1Yjj/\n3neaB97fQF19g9GRHFJdfQO/e38D6/ed4tWJ4Vw7oKPRkYSNkGIXFjEusivPjh3MN9tO8MSnm2mQ\niZBmpbXmyU83889tJ3hmzCDGRcpthsV/yAmuwmISftWT4soaXl+1mwAvE7NvHiiXtJuB1po/rtjO\nR7kFPHjdFSQO62V0JGFjpNiFRT143RWUVNYyf91+An3cuf+avkZHsnt//34v763dT8KVPXho5BVG\nxxE2SIpdWJRSijm3DKKksoZXvt5JgLeJKXE9jI5lt5ZkHuKVr3dya0QXnh4zWD4BiQuSYhcW5+Ki\neGViOGXVdfzhsy34e5m4JayL0bHszvJNx5j92Wau6R/EqxPDcZGhXuIi5OCpsAqTqwtzJw8hukcg\nDy/N54ddhUZHsitrdhXy0NINRPcI5O9TojC5yv+64uLk3SGsxsvdlfkJMfTt4Mu9GbnkHSo2OpJd\nyDtUzD0ZufTt4Mv8hBi83F2NjiRsnBS7sCp/LxPp02Po4OdBUmo2O4/L0LBfsutEOUmp2XTw8yB9\negz+XiajIwk7IMUurK6D77mhYZ4mF+JTMjl8utLoSDbp8OlK4lMy8XBzYVFyHB18ZaiXuDxS7MIQ\n3dp6s3B6HGfrGpiaksnJ8mqjI9mUwvJzQ72qaxvISI6jW1tvoyMJOyLFLgzTv5MvqUkxnCw7S8KC\nbEqrZGgYQGlVLdMWZHGi7CwLEmPo38nX6EjCzpil2JVSC5RSJ5VSW8yxPOE8hnQP5N34KPacLGdG\nejZVNc49NKyqpp6703PYc7Kcd+KjiOoRaHQkYYfMtcWeBow207KEkxneL4jX7owg52Ax9y/Jo9ZJ\nh4bV1jdw/5I8sg+e5q93RDCiX5DRkYSdMkuxa63XAKfNsSzhnG4J68KL40L5bsdJHvloo9MNDWto\n0Dz60Ua+23GSF8aFMCZcLuASLSdXngqbMTmuO8WNowe8TDwz1oBL5r99Do5usOoqNbDvZAUTSqqY\n1dmH7ru8YZdVIwhruvYP0DXKoquwWrErpWYCMwG6d+9urdUKO/Obq/tQfKamaWjYQyP7WTdA5rvg\n7gMB1nuPHi+tpqy0it5+nnT1qYezcm6/Q2uw/K5GqxW71noeMA8gOjrauT5ni8umlGL2zQMpqapt\nGvdr1bG0ugHC7oBRL1hldek/HuDpL7Zy25BgXrk9DCXzX4QZyK4YYXOUUrw0IZSyqlqe+XIbAd7u\n1ruRREM9KOucBfx5/hGe/mIr1w/qyMu3hcpQL2E25jrd8X1gPdBfKVWglEo2x3KF83JzdeGNSZEM\n7d2WRz7ayHc7TlhnxboBlOVnsazecZLff7iRuF5teXNSJG4y1EuYkbnOipmkte6stTZprYO11inm\nWK5wbp4mV96bFs3Azn7ctyiP7ANWOPFKN1h8iz37wGnuW5zLgM6+zE+IxtMkQ72EeclmgrBpvp4m\n0pJi6BroxfS0bLYdLbPsCrVld8VsO1rG9LRsuvh7kZYUi6+nDPUS5ifFLmxeuzYeZCTH0cbDjWkL\nsjhQdMYyK9Lnj+m7WGYL+kDRGaYtyKKNhxsZM+Jo38bDIusRQopd2IWuAV5kJMdS33BuaNiJMgsM\nDdPnT0OzwBb7ibJqpqZkUt/QQEZyLF0DvMy+DiEaSbELu9G3gy/p02MpPlNDfEomJZU15l1Bw/k5\nNWYu9pLKGqalZFF8poa0pFj6dpChXsKypNiFXQkLDuC9adEcKKokKS2bypo68y3cAlvslTV1TE/L\nZn/RGeZNiya8W4DZli3ExUixC7vzq77teWNSJBsPl3BPRi41dWa6kk+bd4u9pq6BexflkX+4hDcm\nRTKsb3uzLFeIS5FiF3ZpdEgnXpoQxtrdRTz8YT715hga1rjFboaDp/UNmlkf5rNmVyEvTQhjdEin\nVi9TiMslV54Ku3VHTDdKqmr444od+HuZeHFcSOuGhplpV4zWmjmfb+Efm47x5I0DuCOmW6uWJ0Rz\nSbELuzZzeB+KK2t5+/u9BHqbePSGAS1fmJkOnv7ln7tYnHmIe0f04Z4RfVq1LCFaQopd2L3HbuhP\nSWUNc1fvJdDbnRlX9W7ZghrPY2/FSIH5a/fx1uo9TIrtxuOj+7d4OUK0hhS7sHtKKV4YF0ppVS0v\nLN9OgLc7t0cFN39BTQdPW7Y755PcAl5Yvp0bQzrxwrhQ68+SF+I8KXbhEFxdFK/dGUF5dQ6Pf7IJ\nP083Rg1u5gHLVuxj/2bbCR77ZBO/7tue1++KwFUmNQoDyVkxwmF4uLnyztQoQrv689v3N7B+76nm\nLaCFZ8Ws33uK+5fkEdLVn3fjo/Bwk6FewlhS7MKh+Hi4kZoYQ4+23ty9MIfNBaWX/+QWHDzdcqSU\nuxfm0L2tN2mJMfh4yIdgYTwpduFwAn3cyUiOw9/LREJqFnsLKy7viU27Yi5vi3tfYQUJC7Lw9zKR\nkRxLoI97CxMLYV5S7MIhdfL3ZNGMOFwUxM/P5GhJ1aWf1Ix97EdLqohPyQIgIzmWzv4y1EvYDil2\n4bB6tfchLSmW8uo64lMyOX3mEkPDLrPYT58fQlZWVUv69Fh6B7UxU2IhzEOKXTi0kK7+zE+IpqC4\nisTULCrO/sLQsMs4eFpxto6k1CwKiquYnxBNSFd/MycWovWk2IXDi+vdjrmTh7D1aBkzF+ZQXVt/\n4Qc2/PKx5URpAAAO4UlEQVR57Gfr6rknI4ctR8uYO3kIcb3bWSixEK0jxS6cwshBHXl1Yhg/7j3F\ngx9soK7+AhMhf2FXTF19Aw++n8+/9pzildvDGDmoo4UTC9FyUuzCaYyPDObpMYP4eusJnlq2Ga3/\nZyLkRc6K0Voze9kWVm49zpxbBjFhSAuuahXCiuSkW+FUkob1oriylje+3U2gtztP3jTwPz+8yDz2\nl1buYGnOYX53bV+m/7qXFdMK0TJS7MLpPDzyCkoqa3h3zT4CvN257+rzExgvsCvmnR/28u4P+4gf\n2oOHr+9nQFohmk+KXTgdpRTPjBlMSWUtL6/cQYC3iUmx3X92VswHWYd46asdjAnvwrNjB8tQL2E3\nzLKPXSk1Wim1Uym1Ryn1hDmWKYQlubgo/nJHOFf3D2L2ss2s2HwMGv6zxf7V5mM8tWwzI/oF8ZeJ\n4bjIUC9hR1pd7EopV2AucCMwCJiklBrU2uUKYWkmVxfenhLFkO6BPPRBPhsPnwZgy7EKHvwgn8ju\ngbwzNQp3NznHQNgXc7xjY4E9Wut9Wusa4APgVjMsVwiL83J3JSUhht5BPry6chsAr63aTe8gHxYk\nxODlLpMahf0xR7F3BQ7/5PuC878nhF3w9zaxcHosAV7nDjn5enmwcHos/t4mg5MJ0TLmKPYL7Xz8\n2S3jlVIzlVI5SqmcwsJCM6xWCPPp4OfJH246d7/UJ24eTAc/T4MTCdFy5ij2AuCnt2EPBo7+74O0\n1vO01tFa6+igoCAzrFYI8+rY5twWeyd/b4OTCNE65ij2bOAKpVQvpZQ7cBfwhRmWK4R1teLWeELY\nklafx661rlNK/Rb4GnAFFmitt7Y6mRDW1jhi4DJvtCGErTLLBUpa6xXACnMsSwjDXGSkgBD2Rq48\nFaJR064YYy9Gqq2tpaCggOrqakNzCON4enoSHByMydSyM7Ok2IVo1DiP/RdutGENBQUF+Pr60rNn\nTxlj4IS01pw6dYqCggJ69WrZ0Dn5zClEIxs5eFpdXU27du2k1J2UUop27dq16hObFLsQjS4yj90I\nUurOrbV//1LsQjSSg6dN2rT5+Q2616xZw5AhQ3Bzc+Pjjz9u1nOFdck7WIhGTac7yv8WF9K9e3fS\n0tKYPHmy0VHEJcg7WIhGNnLw1Fb17NmTsLAwXFwurza01jz66KOEhIQQGhrK0qVLATh27BjDhw8n\nIiKCkJAQ1q5dS319PYmJiU2Pfe211yz5UhyenBUjRCMbOd3xp579civbjpaZdZmDuvjx9JjBZl3m\nhXz66afk5+ezceNGioqKiImJYfjw4SxZsoQbbriB2bNnU19fT2VlJfn5+Rw5coQtW7YAUFJSYvF8\njky22IVoZCNnxTiKdevWMWnSJFxdXenYsSMjRowgOzubmJgYUlNTeeaZZ9i8eTO+vr707t2bffv2\n8cADD7By5Ur8/PyMjm/XZItdiEZNB09tZ1eMNbasLUXrnw15BWD48OGsWbOG5cuXEx8fz6OPPsq0\nadPYuHEjX3/9NXPnzuXDDz9kwYIFVk7sOGTTRIhGssVuVsOHD2fp0qXU19dTWFjImjVriI2N5eDB\ng3To0IG7776b5ORk8vLyKCoqoqGhgdtuu43nn3+evLw8o+PbNdliF6KRFHuTyspKgoODm76fNWsW\nV111FePHj6e4uJgvv/ySp59+mq1bLz7vb/z48axfv57w8HCUUvz5z3+mU6dOpKen88orr2AymWjT\npg0LFy7kyJEjJCUl0XD+vrN/+tOfLP4aHZm62MclS4qOjtY5OTlWX68QvyjrPVjxCDyyG9p0MCzG\n9u3bGThwoGHrF7bhQu8DpVSu1jr6Us+VTRMhGsl57MJByDtYiEZy5alwEPIOFqKR7GMXDkLewUI0\nkmIXDkLewUI0kpECwkFIsQvRSLbYhYOQd7AQjaTYm5w4cYLJkyfTu3dvoqKiuPLKK1m2bFmrlvnM\nM8/w6quvAjBnzhxWrVrVouXk5+ezYkXzb7E8adIkwsLCfjZg7Ke5fmr69Ol06NCBkJCQiy7zYs81\nmryDhWhkQzfaMJLWmnHjxjF8+HD27dtHbm4uH3zwAQUFBT97bF1dXYvW8dxzzzFy5MgWPbclxX78\n+HF+/PFHNm3axMMPP3xZz0lMTGTlypUtiWg4KXYhGskWOwDfffcd7u7u3HvvvU2/16NHDx544AEA\n0tLSmDhxImPGjGHUqFFUVFRw3XXXMWTIEEJDQ/n888+bnvfiiy/Sv39/Ro4cyc6dO5t+PzExselm\nHbm5uYwYMYKoqChuuOEGjh07BsDVV1/N448/TmxsLP369WPt2rXU1NQwZ84cli5dSkRERNMo4EbV\n1dUkJSURGhpKZGQkq1evBmDUqFGcPHmSiIgI1q5de1l/DsOHD6dt27aX/eeWn5/P0KFDCQsLa7pC\nF+CNN95g0KBBhIWFcddddwHwww8/EBERQUREBJGRkZSXl1/2ei6HjBQQolHTwVMbKvavnoDjm827\nzE6hcONLF/3x1q1bGTJkyC8uYv369WzatIm2bdtSV1fHsmXL8PPzo6ioiKFDhzJ27Fjy8vL44IMP\n2LBhA3V1dQwZMoSoqKj/Wk5tbS0PPPAAn3/+OUFBQSxdupTZs2c3DQCrq6sjKyuLFStW8Oyzz7Jq\n1Sqee+45cnJyeOutt36Wa+7cuQBs3ryZHTt2MGrUKHbt2sUXX3zBLbfcQn5+fnP/tC7btGnTePPN\nNxkxYgRz5szh2Wef5fXXX+ell15i//79eHh4NI0jfvXVV5k7dy7Dhg2joqICT09Ps2aRYheikW5w\n+q31C7n//vtZt24d7u7uZGdnA3D99dc3bc1qrXnqqadYs2YNLi4uHDlyhBMnTrB27VrGjx+Pt7c3\nAGPHjv3Zsnfu3MmWLVu4/vrrAaivr6dz585NP58wYQIAUVFRHDhw4JJZ161b1/TJYsCAAfTo0YNd\nu3ZZfAxwaWkpJSUljBgxAoCEhAQmTpwIQFhYGFOmTGHcuHGMGzcOgGHDhjFr1iymTJnChAkT/msu\njzm0qtiVUhOBZ4CBQKzWWgbACPtli8X+C1vWljJ48GA++eSTpu/nzp1LUVER0dH/GVHi4+PT9OvF\nixdTWFhIbm4uJpOJnj17Ul1dDVz6psxaawYPHsz69esv+HMPDw8AXF1dL2t/vhGzry5l+fLlrFmz\nhi+++ILnn3+erVu38sQTT3DzzTezYsUKhg4dyqpVqxgwYIDZ1tnad/EWYAKwxgxZhDCWrnf6A6cA\n1157LdXV1bz99ttNv1dZWXnRx5eWltKhQwdMJhOrV6/m4MGDwLl91MuWLaOqqory8nK+/PLLnz23\nf//+FBYWNhV7bW3tL06MBPD19b3oPunhw4ezePFiAHbt2sWhQ4fo37//L79gM/D39ycwMLBp/31G\nRgYjRoygoaGBw4cPc8011/DnP/+ZkpISKioq2Lt3L6GhoTz++ONER0ezY8cOs+ZpVbFrrbdrrXde\n+pFC2AFb3GI3gFKKzz77jB9++IFevXoRGxtLQkICL7/88gUfP2XKFHJycoiOjmbx4sVNW55Dhgzh\nzjvvJCIigttuu42rrrrqZ891d3fn448/5vHHHyc8PJyIiAh+/PHHX8x3zTXXsG3btgsePP3Nb35D\nfX09oaGh3HnnnaSlpTVt9f+SF154geDg4KYvOHd65JVXXsnOnTsJDg4mJSXlF5eRnp7Oo48+SlhY\nGPn5+cyZM4f6+nqmTp3adDD34YcfJiAggNdff52QkBDCw8Px8vLixhtvvGTG5jDL2F6l1PfAI5e7\nK6bFY3t/eAW2fNz85wlxOSpOQF0NzD5qaAwZ2yugdWN7L7mPXSm1Cuh0gR/N1lp/foHfv9hyZgIz\nAbp37365T/tvbTpAkOU/VgknFdQfOocbnUKIVrtksWutW3YVwc+XMw+YB+e22Fu0kKiEc19CCCEu\nSnYoCiGEg2lVsSulxiulCoArgeVKqa/NE0sI52aLp+0J62nt33+rzmPXWi8DWjcZSAjxXzw9PTl1\n6hTt2rW75HngwvForTl16lSrrkaVK0+FsDHBwcEUFBRQWFhodBRhEE9Pz1ZdjSrFLoSNMZlM9OrV\ny+gYwo7JwVMhhHAwUuxCCOFgpNiFEMLBmGWkQLNXqlQhcNDqK2699kCR0SGsTF6z43O21wv2+5p7\naK2DLvUgQ4rdXimlci5nToMjkdfs+Jzt9YLjv2bZFSOEEA5Gil0IIRyMFHvzzDM6gAHkNTs+Z3u9\n4OCvWfaxCyGEg5EtdiGEcDBS7C2glHpEKaWVUu2NzmJpSqlXlFI7lFKblFLLlFIBRmeyFKXUaKXU\nTqXUHqXUE0bnsTSlVDel1Gql1Hal1Fal1INGZ7IWpZSrUmqDUuofRmexBCn2ZlJKdQOuBw4ZncVK\nvgFCtNZhwC7gSYPzWIRSyhWYC9wIDAImKaUGGZvK4uqA32utBwJDgfud4DU3ehDYbnQIS5Fib77X\ngMcApzg4obX+p9a67vy3/wZaPnLOtsUCe7TW+7TWNcAHwK0GZ7IorfUxrXXe+V+Xc67ouhqbyvKU\nUsHAzcB8o7NYihR7MyilxgJHtNYbjc5ikOnAV0aHsJCuwOGffF+AE5RcI6VUTyASyDQ2iVW8zrmN\nswajg1iKjO39H790827gKWCUdRNZ3uXcsFwpNZtzH90XWzObFV3ojhZO8alMKdUG+AR4SGtdZnQe\nS1JK3QKc1FrnKqWuNjqPpUix/4+L3bxbKRUK9AI2nr+rTTCQp5SK1Voft2JEs7vUDcuVUgnALcB1\n2nHPjy0Auv3k+2DgqEFZrEYpZeJcqS/WWn9qdB4rGAaMVUrdBHgCfkqpRVrrqQbnMis5j72FlFIH\ngGittT0OErpsSqnRwF+BEVprh72lj1LKjXMHh68DjgDZwGSt9VZDg1mQOreFkg6c1lo/ZHQeazu/\nxf6I1voWo7OYm+xjF5fyFuALfKOUyldKvWN0IEs4f4D4t8DXnDuI+KEjl/p5w4B44Nrzf7f557dk\nhZ2TLXYhhHAwssUuhBAORopdCCEcjBS7EEI4GCl2IYRwMFLsQgjhYKTYhRDCwUixCyGEg5FiF0II\nB/P/7QFP6lIco78AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10ffb8940>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = gluon.loss.L1Loss()\n",
    "\n",
    "# getting data ready\n",
    "output = nd.arange(-5,5,0.01)\n",
    "output.attach_grad() # we need the gradient\n",
    "thelabel = nd.zeros_like(output) \n",
    "with ag.record():    # start recording\n",
    "    theloss = loss(output, thelabel)\n",
    "theloss.backward()   # and compute the gradient\n",
    "\n",
    "plt.plot(output.asnumpy(), theloss.asnumpy(), label='L1 loss')\n",
    "plt.plot(output.asnumpy(), output.grad.asnumpy(), label='Gradient of L1 loss')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we move on to other losses, let's quickly consider what happens if we want to minimize the L1 loss. Consider the toy example where we have a number of labels $y_i$ and we want to fit *all* of them to a single scalar, say $f$. In this case we need to solve the minimization problem:\n",
    "\n",
    "$$\\mathop{\\mathrm{minimize}}_f \\sum_i |y_i - f|$$\n",
    "\n",
    "As we saw above, the gradient is either -1 or 1. Hence, for the gradients to the left and to the right of $f$ to cancel out we need *the same number of $y_i$* on either side. This is the definition of the *median*. Hence, minimizing the L1 loss means that we are computing the median (at least for constant predictions). In general, the L1 loss is very robust against outliers, since the gradients can never get too large. \n",
    "\n",
    "## L2 loss\n",
    "\n",
    "Taking the squared distance between observation and estimate tends to be the default choice in many problems. Often for convenience we multiply this loss by a factor of $\\frac{1}{2}$ to ensure that the derivatives look pretty. Here's the loss:\n",
    "\n",
    "$$l(y,f) = \\frac{1}{2} (y-f)^2$$\n",
    "\n",
    "For vectorial $f$ and $y$ this is the squared Euclidean distance between points. The L2 loss has a few other nice properties. By a similar argument as before we can see that $\\sum_{i=1}^m \\frac{1}{2} (y_i - f)^2$ is minimized by choosing $f = \\frac{1}{m} \\sum_{i=1}^m y_i$, i.e. by choosing the mean. Let's see what it looks like in practice."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd0VVX2wPHvSScNCCS0AKGGJJAECE2UIr0j6NBUUGcQ\ndexjxe7oONgLOjJ2pf4QBBVFUBRQQAIkJKETWhIIKaQX8pLz++NGZCAh7SX3JW9/1mLllfvu3S9k\nnX3vuefso7TWCCGEsD8OZgcghBDCHJIAhBDCTkkCEEIIOyUJQAgh7JQkACGEsFOSAIQQwk5JAhBC\nCDslCUAIIeyUJAAhhLBTTmYHcCXNmzfXAQEBZochhBD1xq5du1K11r6V2damE0BAQACRkZFmhyGE\nEPWGUupEZbeVLiAhhLBTkgCEEMJOSQIQQgg7JQlACCHslCQAIYSwU5VOAEqpj5RSZ5VSsRe99rJS\n6oBSaq9SarVSqkk5nz2ulIpRSkUppWRYjxBC2ICqXAF8Aoy+5LUNQHetdShwCHjsCp8fqrUO11pH\nVC1EIYQQtaHSCUBrvRlIv+S1H7TWltKn2wF/K8ZWLYWWYhZtPsrO4+kVbyyEEDYm/3wxluKSOjmW\nNe8B3Ap8V857GvhBKbVLKTX3SjtRSs1VSkUqpSJTUlKqHERJCXy09Tj//HY/st6xEKK+efPHw4x+\ncwsFRcW1fiyrJACl1HzAAiwuZ5OBWutewBjgLqXUoPL2pbVepLWO0FpH+PpWajbz/2jk4siDI7sS\nfSqDb/aervLnhRDCLEkZ+Xz86zF6tGmMm7NjrR+vxglAKTUbGA/M0uWccmutk0p/ngVWA31retwr\nmdLLn24tvViw/gCFltrPokIIYQ2v/HAQDTw4smudHK9GCUApNRp4BJiotc4rZxsPpZTXH4+BkUBs\nWdtai6OD4vGxQZxKz+fzbZUuiyGEEKaJTcxk9Z5Ebh3YAf+m7nVyzKoMA10KbAMClVIJSqnbgHcA\nL2BD6RDP/5Ru21opta70oy2ArUqpaOB34Fut9fdW/RZlGNTVl2u6NOftn46QmVdU24cTQohq01rz\n4rr9NGnkzJ1DO9XZcStdDVRrPaOMlz8sZ9skYGzp43ggrFrR1dDjY4MY+9YWFv58hMfHBpkRghBC\nVOjngyn8djSNZyYE4+3mXGfHbdAzgYNaeTO1lz+f/HqcU+ll9lAJIYSpLMUlvLhuPwHN3JnZr32d\nHrtBJwAwbqY4OBg3V4QQwtb8364EDp/N4dEx3XBxqtsmucEngFaNG3Hb1R1YE5VETEKm2eEIIcQF\nuYUWXv3hEBHtmzIqpGWdH7/BJwCAeYM70czDhRfXyeQwIYTtWLQ5ntScQh4fF4RSqs6PbxcJwMvN\nmXuHd2FbfBqbDp41OxwhhCA5q4BFm+MZF9qKXu2amhKDXSQAgBl929GxuQcvfLufojqqsyGEEOV5\nfcMhLCUlPDKqm2kx2E0CcHZ0YP64II6m5PLFdpkcJoQwz76kLFZEnuKm/gG0a1Y3k77KYjcJAODa\nbn5c06U5b2w8zLnc82aHI4SwQ1prnvsmjsaNnLl3WBdTY7GrBKCU4snxweQUWnh94yGzwxFC2KHv\nY8+wPT6dB0YG0ti97iZ9lcWuEgBA1xZezOrXjsU7TnIoOdvscIQQdqSgqJgX1u2nW0svZvRpa3Y4\n9pcAAO4f3hVPVyee/2afDAsVQtSZD7ceI+FcPk+ND8bJ0fzm1/wITNDUw4X7hndhy+FUftwvw0KF\nELUvOauAhZuOMCqkBVd1bm52OICdJgCAG/u3p5OvBy+s2895iwwLFULUrn9/fwBLsWb+2GCzQ7nA\nbhOAs6MDT4wP5lhqLp/+dtzscIQQDVjUqQxW7U7ktms6mDrs81J2mwAAhgb6MSTQl7d+PExqTqHZ\n4QghGiCtNc9+HYevlyt3De1sdjj/w64TAMAT44LJLyrm1R9kWKgQwvrWRCWx52QGD48KxNO10kuw\n1Am7TwCd/Ty5eUAAy3aeZG9ChtnhCCEakNxCCy99d4BQ/8ZM7eVvdjiXsfsEAHDfiC4093TlyTVx\nlJTIsFAhhHW8/dMRzmQV8PSEYBwc6r7aZ0UkAQDebs48PrYb0acyWBF5yuxwhBANwJGz2XywJZ4b\nevvTu72P2eGUqUoJQCn1kVLqrFIq9qLXfJRSG5RSh0t/llnXVCk1u3Sbw0qp2TUN3Nomh7ehb4AP\n//7+ABl5UidICFF9WmueXhuHu4sjj4wxr9pnRap6BfAJMPqS1x4FftRadwF+LH3+P5RSPsDTQD+g\nL/B0eYnCLEopnpscQlaBhZfXy/KRQojq+zbmNL8eSeOhUYE093Q1O5xyVSkBaK03A+mXvDwJ+LT0\n8afA5DI+OgrYoLVO11qfAzZweSIxXbeW3sweEMCS30/K8pFCiGrJKbTw/Df7CGntXeeLvFeVNe4B\ntNBanwYo/elXxjZtgIs71xNKX7uMUmquUipSKRWZkpJihfCq5r4RXWjm4cqTa2LlhrAQosre/vEw\nyVmFPD+5O442eOP3YnV1E7is30KZravWepHWOkJrHeHr61vLYV3O282Z+eO6EXUqg//bJTeEhRCV\ndzg5mw+3HmNaRFvTlnmsCmskgGSlVCuA0p9lVVdLAC6ufeoPJFnh2LXijxvCL30nN4SFEJWjteap\nNcaN34dHB5odTqVYIwGsBf4Y1TMbWFPGNuuBkUqppqU3f0eWvmaTlFI8O0luCAshKu/rvafZFp/G\nQ6O70cyGb/xerKrDQJcC24BApVSCUuo24CVghFLqMDCi9DlKqQil1AcAWut04HlgZ+m/50pfs1lB\nrf68Ibz75DmzwxFC2LCsgiL++c0+urfxZmbfdmaHU2nKlhdEiYiI0JGRkaYdP6fQwojXfqFxI2e+\nvvtqnG1gAQchhO158qtYFu84weo7BxLWtompsSildmmtIyqzrbRoV+Dp6sSzE0M4cMa4sSOEEJfa\ndeIcX+w4wc0DAkxv/KtKEkAFRoa0ZGRwC97YeIhT6XlmhyOEsCFFxSXMXx1DS283/jGqftz4vZgk\ngEp4dlIIjkrxxFexsoawEOKCD7Yc48CZbJ6dGGJzpZ4rQxJAJbRq3Ih/jArkl0MpfLP3tNnhCCFs\nwMm0PN788RAjg1swMqSl2eFUiySASrp5QACh/o159ut9ZOYXmR2OEMJEWmvmfxWDk4MDz04KMTuc\napMEUEmODooXr+tBem4h//7+gNnhCCFMtDY6iS2HU/nHyK60atzI7HCqTRJAFXRv05hbB3ZgyY6T\nRB636WkMQohakpF3nue/2UdY2ybcNCDA7HBqRBJAFd0/oittmjTi0VUxFBQVmx2OEKKO/WvdAc7l\nFfHidbZf7K0ikgCqyMPViRen9ODI2Rze/umw2eEIIerQlsMpLI88xV+v6UBI68Zmh1NjkgCqYXBX\nX67v7c9/foknNlHWDRDCHuQUWnj0yxg6+npw//CuZodjFZIAqunJccH4eLjw0Mq9FBWXmB2OEKKW\nLfj+AEmZ+SyYGoqbs6PZ4ViFJIBqauzuzAuTu7P/dBb/+fmo2eEIIWrRjvg0Ptt2gjlXBRARYJsL\nvFeHJIAaGBnSkglhrXnrp8McSs42OxwhRC3IP1/Mw1/upZ2POw/Vw3IPVyIJoIaemRCMl5szD/1f\nNBbpChKiwXn1h4OcSMvjpak9cHepf+UerkQSQA0183TlmYkhRCdk8tGvUjFUiIZk98lzfPjrMWb1\na8dVnZqbHY7VSQKwggmhrRgR3IJXfzhEfEqO2eEIIaygoKiYh1fupZW3G4+O6WZ2OLVCEoAVKKV4\nYXJ3XJ0c+Id0BQnRILyx8TBHzubwr6mheLk5mx1OrZAEYCV+3m48P7k7u09m8P7meLPDEULUwM7j\n6by/+SjT+7RlcFdfs8OpNZIArGhiWGvGhbbijY2HiEuSCWJC1Ee5hRYeXBGNf9NGPDE+2OxwalWN\nE4BSKlApFXXRvyyl1H2XbDNEKZV50TZP1fS4tkgpxT8ndaeJuwsPLI+m0CK1goSob15Yt59T5/J4\n9YbwernIS1XUOAForQ9qrcO11uFAbyAPWF3Gplv+2E5r/VxNj2urmnq4sGBqKAeTs3ltwyGzwxFC\nVMGmg2dZsuMkc6/pSN8ODWfCV3ms3QU0DDiqtT5h5f3WK0O7+TGjbzsWbY5np5SNFqJeOJd7nkdW\n7iWwhRf3j2gYtX4qYu0EMB1YWs57A5RS0Uqp75RS9XcJnUp6YlwQbZu68+CKaHILLWaHI4SowJNr\nYjmXd57XpoU1mFo/FbFaAlBKuQATgf8r4+3dQHutdRjwNvDVFfYzVykVqZSKTElJsVZ4dc7D1YlX\nbgjj1Lk8Xli33+xwhBBXsDY6iW/2nua+4V0bRJnnyrLmFcAYYLfWOvnSN7TWWVrrnNLH6wBnpVSZ\n0+q01ou01hFa6whf3/o9/KpvBx/mXtORJTtOsnHfZb8WIYQNSMzI54nVMfRs14TbB3U0O5w6Zc0E\nMINyun+UUi2VUqr0cd/S46ZZ8dg264GRXQlu5c1DK6NJziowOxwhxEWKSzT3L4uiuETz+l/CcXK0\nr5HxVvm2Sil3YASw6qLX5iml5pU+vR6IVUpFA28B07XW2hrHtnWuTo68NaMnBUUlPLgimpISu/ja\nQtQLCzcd4ffj6Tw/uTsBzT3MDqfOWSUBaK3ztNbNtNaZF732H631f0ofv6O1DtFah2mt+2utf7PG\nceuLzn6ePD0hmK1HUlm0RWYJC2ELdp1I580fDzM5vDVTevmbHY4p7Ot6x0TT+rRlTPeWvLL+INGn\nMswORwi7lplfxD1Lo2jdxCjhYq8kAdQRpRQvTQnFz8uVe5ftIUeGhgphCq0181fHcCargDen92yw\nhd4qQxJAHWrs7swb03tyMj2Pp9fEmR2OEHZp5a4Evtl7mgdGdKVXu6Zmh2MqSQB1rG8HH/4+tDNf\n7k5gTVSi2eEIYVfiU3J4em0c/Tv6MG9wJ7PDMZ0kABPcM6wLEe2b8viqGFlARog6UlBUzF1L9uDi\n5MDr08JxdFBmh2Q6SQAmcHJ04K0ZPXFxcuDOxbspKJKqoULUtme/jmP/6Sxe/0s4rRo3MjscmyAJ\nwCStmzTi9WnhHDiTLfcDhKhlq3YnsPT3U9w5pBNDu/mZHY7NkARgoiGBfvx9aGeWR55i5a4Es8MR\nokE6nJzN/NWx9O3gwwN2UuWzsiQBmOy+4V3o39GHJ76K4eCZbLPDEaJByTtv4Y7Fu/FwdeTtGT3t\nrtRDReS3YTInRwfemt4TT1dn7ly8S0pHC2ElWmueWB3L0ZQc3pzekxbebmaHZHMkAdgAP2833poR\nzrHUXB5fHYOdlEkSolYt33mKVXsSuW9YVwZ2LrP4sN2TBGAjrurUnPuHd2VNVBKfb7frBdWEqLG9\nCRk8tTaOa7o05+/XdjY7HJslCcCG3DW0M8O6+fHc1/vYEW8X1bKFsLrUnEJu/3wXvp6uvDm9p4z3\nvwJJADbEwUHx+vRw2vm4c9eS3ZzOzDc7JCHqlaLiEu5avJv03PO8f1NvfDxczA7JpkkCsDHebs4s\nurk3+eeLmfeFTBIToipeXLefHcfSeWlqD7q3sZ+lHatLEoAN6uznxat/CSf6VAZPrYmVm8JCVMKq\n3Ql8/Otxbh3Yget62md9/6qSBGCjRndvyd3XdmZFZAJf7DhpdjhC2LTYxEweWxVD/44+PDa2m9nh\n1BuSAGzY/cO7MjTQl2fXxrHzeLrZ4Qhhk9JKb/o283Bh4cxeOMtkr0qT35QNc3BQvDG9J2193Jn3\n+S5OpeeZHZIQNqXQUsy8L3aRmlPI+zdF0MzT1eyQ6hWrJQCl1HGlVIxSKkopFVnG+0op9ZZS6ohS\naq9Sqpe1jt2QNW7kzAezIygqLuG2T3eSXVBkdkhC2AStNY+vimXn8XO8ckMYPfzlpm9VWfsKYKjW\nOlxrHVHGe2OALqX/5gLvWfnYDVYnX0/eu7E3R1NyuXvpHizFJWaHJITp3vvlKF/uTuC+4V2YENba\n7HDqpbrsApoEfKYN24EmSqlWdXj8em1g5+Y8P6k7Px9M4YV1+80ORwhTfR97hgXfH2RiWGvuHdbF\n7HDqLWsmAA38oJTapZSaW8b7bYBTFz1PKH1NVNLMfu24dWAHPv71uJSLEHYrNjGT+5dHEd62CQuu\nD0WpBjTTN+Mk/LIA1txVJ4dzsuK+Bmqtk5RSfsAGpdQBrfXmi94v63/psgHupcljLkC7du2sGF7D\nMH9cEMfTcnlmbRwBzdy5pouv2SEJUWeSswr466eRNHU3Jky6OTuaHVLNFWbDvrUQvRSObzFe6zAI\nLOfBqXZnMlvtCkBrnVT68yywGuh7ySYJQNuLnvsDSWXsZ5HWOkJrHeHrK43bpRwdFG/N6EkXP0/u\nXLxb1hAQdiO7oIhbPt5JVkERH87pg59XPS7vXFIC8T/Dqtvhla6w5k7ISoSh8+HevTD761pv/MFK\nCUAp5aGU8vrjMTASiL1ks7XAzaWjgfoDmVrr09Y4vr3xdHXig9kRNHJ2ZM7Hv0vNINHgnbeUcMcX\nuzmYnM27s3oR1Mrb7JCqJ/UwbHwW3ugBn02Cg99B6F/g1h/g7t0w+GFo2r7OwrFWF1ALYHVpX5wT\nsERr/b1Sah6A1vo/wDpgLHAEyANusdKx7ZJ/U3c+vqUP097fzpyPdrJi3gAaN3I2OywhrE5rzaNf\n7mXrkVRevj6UIYH1bE3f/HMQ+yVELYXESFAO0GkYjHwOAseCs3kL1CtbrjMTERGhIyMvm1IgLvLr\nkVTmfPw7vdo15dNb+zaMPlEhLrLg+wO8+/NRHhzRlbvry4if4iI48iNELzHO8ovPg18whM0wzvi9\nWtbaoZVSu8oZin8Za94EFiYY2Lk5r9wQxr3LonhwRTRvz+iJg9Q/Fw3EZ9uO8+7PR5nRt139WNjl\nTIxxph+zAnJTwL0ZRNxqNPytwsDGRixJAmgAJoW3ITmrgBfXHcDP25Wnxgc3rKFxwi59H3uGp9fG\nMTzIj+cnhdju33TOWdi7AqKXQXIMODhD4GgImwmdh9fJzdzqkgTQQPztmo6cySzko1+P0dzTlbuG\n1oOzJSHK8euRVO5Zuocw/ya8PaMXTrZW4K2oAA59Z5ztH9kIuhha94Kxr0D3qeDuY3aElSIJoIFQ\nSvHEuCDScwt5ef1BPF2dmH1VgNlhCVFlu06c42+fRdKhuQef3NKHRi42cl9La0iINPr1Y7+Egkzw\nagVX3W108fjVvzLUkgAaEAcHxcs3hJF7vpin18bh4erE9b1lYQxRf+xLyuKWj3/H18uVz2/rSxN3\nG+g+yUwwuneil0LaEXBqBEHjjUa/4xBwsJEEVQ2SABoYZ0cH3p7Rk79+GsnDK6PxcHFkTA8puSRs\nX3xKDjd/tAMPVye+uK0fft4mTvQ6n/vn7NxjmwEN7QfCwPsgeBK41dN5CJeQBNAAuTk7sujm3tz0\n4e/cs2wPH7g6MbirzKoWtisxI58bP9iB1vD5bf1o6+Ne90GUlMCJrUa//r41UJQLTQNgyKMQOg18\nOtR9TLVMEkAD5e7ixEdz+jBj0XZu/zyST2/pS7+OzcwOS4jLJGcVcOMHO8gutLD0b/3p7OdZtwGk\nHTXO9KOXQ+ZJcPGC7lMgfCa0G2BzQzetSSaCNXCpOYVMe38bpzML+HhOH0kCwqYkZxUwfdF2zmYV\n8Nltfendvo5Gz+RnQNxqo+E/tcOYndtxiDF0s9s4cDHhCsRKqjIRTBKAHTibXcDM/+4g8Vw+H9/S\nh/6SBIQNqPPGv9gCR38yGv0D30JxIfh2+3N2rnfDWFRGEoC4TEp2ITP+u53Ec/l8NKcPAzpJEhDm\nqdPGPzkOopZAzP9BTjI0ago9bjAa/tY9G1wXjyQAUaaU7EJm/nc7Cefy+XBOBFd1am52SMIO1Unj\nn5tqNPhRS+DMXnBwgi6jIHyG8dOGZ+fWlCQAUa6U7EJmfbCdk+l5fDi7DwM7SxIQdScpI59ZH+yo\nncbfUgiH1htdPId/gBKLUX8nbCb0uB487ONvXRKAuKLUnEJm/XcHx9JyWTizFyOCW5gdkrAD8Sk5\n3PTh72TlF/HJrX2s0/hrDYm7jUY/dqVRetmzpdGnHzYDWgTX/Bj1jFQDFVfU3NOVZXP7M+eTncz7\nYhcvXx/KlF4yY1jUnn1JWdz8kTHOf+nc/nRv07hmO8xKKp2duwxSD4KTmzF6J2ymMZrHUZq2ypDf\nkp1q6uHC4r/2Y+5nkTywIpqs/CLmDGx4E12E+XadSGfOxzvxdHXii7/2o5NvNcf5n8+DA98Y/frx\nPwMa2vaHCW9C8GRo1MSaYdsFSQB2zNPVmCx2z9I9PPP1PrIKLNx9bWfbLbsr6p0th1OY+9kuWjZ2\n4/Pb+uLftIrj60tK4OQ2owBb3Bo4nw2N28GghyBsOjTrVDuB2wlJAHbOzdmRd2f14pEvY3htwyHO\n5Z3nyXHBsqiMqLGv9iTy0MpoOvl68vlt/fD1cq38h9PjjZm50Ush4wS4eBpn+WHTjZo8DjZWHrqe\nkgQgcHJ04OXrQ2ncyJmPfj3GmcwCXp8WLstLimrRWvPuz0d5ef1B+nXwYdFNETR2r8R61QWZEPeV\n0eif3AYo6DgYhs43qm+6eNR67PZGEoAAjFLST00Ipk3TRvzz230k/3c7/705gmaeVThrE3bPUlzC\n02vjWLzjJBPDWvPyDaG4Ol3hRKKkGOI3GQXYDnwDlgJo1gWGPWUUYGssgxNqU40TgFKqLfAZ0BIo\nARZprd+8ZJshwBrgWOlLq7TWz9X02ML6bru6A22auHHvsiimvPcbn9zSlw7N5cxLVCzvvIW7l+zh\nxwNnmTe4Ew+PCiy/K/HsAaNff+8KyD4Nbk0gfJZRgK1N7wY3O9dW1XgegFKqFdBKa71bKeUF7AIm\na633XbTNEOAfWuvxVdm3zAMwzx+rMmmtWXRzBH0C6scSd8IcpzPzmfvZLuKSMnl2Undu6t/+8o1y\n04yVtKKXQNIeUI7QZaTRrx84BpzkatMa6nQegNb6NHC69HG2Umo/0AbYd8UPCpvWu31TVt1xFbd8\nspOZ/93OPyd3Z1qfdmaHJWzQ7pPnuP3zXeQVWlh0UwTDL55YaDlvzMqNXmrM0i0pgpY9YNS/jHo8\nnrJOhZmseg9AKRUA9AR2lPH2AKVUNJCEcTUQV84+5gJzAdq1kwbHTAHNPVh951XcvXQPj3wZw76k\nLJ4YH4yzrS3QLUyzclcCj6+KoWVjNxb/tR9dW3gZs3NPRxn9+rErIS8NPPyg3+3G7NyW3c0OW5Sy\nWikIpZQn8AvwgtZ61SXveQMlWuscpdRY4E2tdZeK9ildQLbBUlzCv78/wH+3HKN/Rx/endUbH4+G\nW0xLVMxSXMJL3x3gg63HuKpTMxbO7EXTknTYu9xo+FP2g6MLBI41+vU7DZPZuXWkzmsBKaWcgW+A\n9Vrr1yqx/XEgQmudeqXtJAHYllW7E3h0VQy+nq4surk3Ia1rOJ1f1EtpOYXctzyKLYdT+Wv/Vjza\n4ShOMcuMWvu6BPz7Gv363acYpZdFnarTewDKmDb6IbC/vMZfKdUSSNZaa6VUX8ABSKvpsUXdmtLL\nn06+ntz++S6ue/c3npkQwoy+bWXmsB3ZeTyduxfvpkN+DD92iaHT/g0QlQXe/nD1A0YXT/POZocp\nKska12QDgZuAGKVUVOlrjwPtALTW/wGuB+5QSlmAfGC6tuUypKJcYW2b8M09V3P/8igeXx3D9vg0\nXpzSA09XubxvyLTWLP1hK8lbPuFL5620cToDZzwgeKLR6AdcI7Nz6yEpBy2qpaRE8+7PR3htwyEC\nmnmwcFYvglp5mx2WsLbCbPKiVnFq04cEFkQDYGl3NU69ZkHQRHCt4wXcRYVkPQBRZ7YdTeOeZXvI\nyi/iifHB3NivnXQJ1XclxXBsM0QvpThuLY7F+RzTLTnXeSo9x9+OalrGGH9hMyQBiDqVkl3IAyuM\nm4JDAn1ZMDUUP283s8MSVZVy6M/ZuVmJFDh68uX5fvzmOZK5M6YR1k5u6NYHkgBEndNa8/n2E7y4\nbj9uzo68eF0PxvZoZXZYoiJ56aWzc5dC4i5QjuS0HcxbaX34NC2I6/t1Zv64INxd5B5PfSEJQJjm\naEoODyyPIjohk+t6tuGZCSGVqwQp6k5xERzZaCyscuh7KD4PfiEUh07ns9y+/GvLObxcnVhwfSjD\ngmS50PpGloQUpunk68nKO67inZ+O8M6mI2w5nMozE4MZ16OV3Bsw2+m9xpl+zP9Bbgq4N4eI2yB8\nBrHF7Xn4yxj2nU5jTPeWPDepe9Xq94t6Sa4ARK2JTczk0VV7iU3MYlg3P56b3J02TRqZHZZ9yU6G\nmBXG2rnJseDgDIGjjbVzu4ygoMSBN388zKLN8fh4uPD8pBBGd5euu/pMuoCEzbAUl/DJb8d59YdD\nOCh4cGQgNw1oL/WEalNRARxcZ5ztH/kRdLFRYjlsBnSfCu5GZdcf9yfz3Df7OJGWx7SItjw+Nki6\n6xoASQDC5pxKz+OJr2L55VAKXfw8eXpCCFd3aW52WA2H1pCw0+jXj1tlrK7l1RrCphkNv2/ghU2P\npeby3NdxbDqYQidfD56b1J2BneX/oqGQBCBsktaaH/Yl88K3+zmZnseI4BY8MS6I9s1kwZlqyzgF\ne5cZXTxpR8CpEQRNgPAZ0GEwOPy5GldOoYV3Nx3hgy3HcHFy4N5hXZh9VQAuTnI11pBIAhA2raCo\nmA+3HmPhpiNYijWzr2rPHUM6S4XRyirMgf1fG2P2j20BtLFQetgMCJ4Ebv87I7vQUsySHSd556cj\npOWeZ0qvNjw6upvM1WigJAGIeiE5q4CX1x9k1e4E3F2c+Os1Hbjt6g54uUk/9GVKSuD4FqNff99a\nKMqFpgHGzdywacbjyz6iWRudxCs/HCThXD4DOjbjkTHdCG/bpM7DF3VHEoCoVw4nZ/PahkN8F3uG\npu7O3DGkE7P6tcdDCsxB2lGjX3/vcsg8Ba7eEDLZaPjb9S9z7dziEs23Mad5d9MRDpzJJriVN4+M\n6cagLs0GANLFAAAZsklEQVRlKK4dkAQg6qW9CRm8vP4gWw6n0riRM7OvCmDOVQH21zWUn2HcyI1a\nCgm/g3KAjkONhVW6jQPnsofSnreU8NWeRN775SjHUnPp5OvBPcO6MCG0dfmLs4sGRxKAqNd2nzzH\nez8fZcO+ZBo5OzKtT1tuu7oDbX3czQ6t9hRbjAVVopfAgXVQXAi+3Yx+/dBp4F3+2PzMvCJWRJ7i\n41+PkZRZQEhrb/4+tDOjQlpKw2+HJAGIBuFwcjb/+SWeNVGJFGvNtYF+3DigPYO7+Dachu1MrNGv\nv3cF5J6FRj7GYunhM6BVeJldPH/YfzqLz7YdZ/WeRAqKSugb4MMdQzsxpKuvdPXYMUkAokFJyshn\n6e8nWfr7KVJzCmnr04iZfdszpVcbWtTHkSw5KUY5huglcCYGHJyg62jjbL/LSHAqv8srp9DCupjT\nrIxM4Pfj6bg5OzA5vA03DWgvS3QKQBKAaKDOW0pYH3eGz7ef4Pdj6SgFAzs1Z1J4a0Z3b2nbo4cs\nhUbhtailcGQDlFiMM/zwmdD9evBoVv5Hi0vYHp/Ol7sT+D72DPlFxXRo7sH0Pm2Z1qctTdzt7B6J\nuCJJAKLBi0/J4auoJL7ak8jJ9DxcnRy4tpsfw4NaMLSbn23cONYaEncbZ/oxK6EgAzxbQuhfjIbf\nL6jcjxZaivn1SCrrY5PZsD+Z9NzzeLk5MSGsNVN7+dOrXRPp5hFlkgQg7IbWmj2nMvhqTyLr486Q\nnFWIg4Le7ZtybbcWXNWpGSGtvXGqy9pDmYl/zs5NPQRObtBtfOns3CHgePnwVq01x9Py2Hokla2H\nU/j1SBo5hRa8XJ24NsiPUSEtubabH27OjpcfT4iL1HkCUEqNBt4EHIEPtNYvXfK+K/AZ0BtIA6Zp\nrY9XtF9JAKIqtNbEJmaxYX8yG/cls+90FgBerk706eBD/44+hLdtSkhrb+vPMTifC/u/Mc72438B\nNLQbYPTrh0wGt//tny8qLuHgmWz2nMog6mQG2+PTSMzIB6BNk0YM6urLqJAWDOjUDFcnafRF5dVp\nAlBKOQKHgBFAArATmKG13nfRNncCoVrreUqp6cB1WutpFe1bEoCoibPZBWyPT2fb0TR2xKcRn5oL\nGANrOjb3INS/CV1beNGhuQcdfT1o5+NetTPskhI4+ZvRr7/vKzifA03aGY1+2HTw6UhxieZMVgFH\nzuZwODmbI2dzOJScTVxSFoWWEgCae7oQ0d6Hq7s05+rOzWnfzF26d0S11XUCGAA8o7UeVfr8MQCt\n9b8u2mZ96TbblFJOwBnAV1dwcEkAwprOZhcQm5hJTEIWMYkZxCRmkpxVeOF9pcDPyxU/Lzf8vFzx\n9XKluacr7q6OeLg40cjFEXcXRzxyTtH65Ff4n1yDR14iRY7uHPUdzh6fMRxw6U5qnoUzmQWczsgn\nObuQ4pI//8ybujvTxc+L7m0a07NdE8LbNsG/aSNp8IXV1PWKYG2AUxc9TwD6lbeN1tqilMoEmgGp\nl+5MKTUXmAvQrl07K4QnhMHPy41ru7lxbbc/lznMLijieGoex9JyiU/JIfFcPmezC0nKLCA6IZO0\n3EK0Bi/yGOe4nSmOW+jrcJASrdha0p0viyfxQ0kE+bluOJwALzejnEWrxo3o36kZrRq70apxIzr7\nedLFz5NmnrLKlrAd1kgAZZ26XHpmX5ltjBe1XgQsAuMKoGahCXFlXm7O9PBvTA//MsbQlxSjj/5E\n8Z4lOB5ah7IUUNikMymBj5ITOAV/rzbcpxSPOTvi5eaEu4ujnMmLesUaCSABaHvRc38gqZxtEkq7\ngBoD6VY4thDWd3Z/aQG2FaicMzi5NYGeN0LYTFzb9MJXKXzNjlEIK7BGAtgJdFFKdQASgenAzEu2\nWQvMBrYB1wM/VdT/L0Sdyk2D2JVGw386ypid23mEMXSz62hwkq4b0fDUOAGU9un/HViPMQz0I611\nnFLqOSBSa70W+BD4XCl1BOPMf3pNjytEjVnOw+H1xnj9Q+uhpAhahsLol4zZuZ5yni8aNqsMhtZa\nrwPWXfLaUxc9LgBusMaxhKgRrSFpj1GALWYl5KeDhx/0u92YndsixOwIhagzsuKGsA9Zp41FVaKX\nQsoBcHSFbmONhVU6XVvm7FwhGjr5qxcNV1E+HPjW6NeP3wS6BPz7wvjXIeQ6aNTU7AiFMJUkANGw\naA0ntxln+nFfQWEWNG4L1zwIodOheWezIxTCZkgCEA3DuePGzdzopcZjZw8InmSM4ml/NTjUYTE4\nIeoJSQCi/irIgn1rjEb/xK+Agg7XwOBHIWgCuHqaHaEQNk0SgKhfSorh2C9GAbb9X4MlH3w6wbVP\nGF08TdpWvA8hBCAJQNQXKQeNM/3o5ZCdZJRXDp9hVN7073PFtXOFEGWTBCBsV146xH5pjOJJ2g3K\nEToPh9EvQtcx4FwP1wMWwoZIAhC2pbgIDm8wFlY5+L0xO7dFdxj5AvS4AbxaVLwPIUSlSAIQ5tMa\nzuw1+vVj/g/yUsG9OfT9m9HF0yrU7AiFaJAkAQjzZJ8xGvyopXA2DhxdIHCM0eh3Hg6OzmZHKESD\nJglA1K2iAjj4rdHoH/3RmJ3bJgLGvQohU8Ddx+wIhbAbkgBE7dMaTv1u9OvHrobCTPBuAwPvM872\nfbuaHaEQdkkSgKg9GSeNYZvRSyH9KDi7GxO0wmZAh0HgUIUF2IUQVicJQFhXYQ7sX2sM3Ty+xXgt\n4Bq45gGjNIOrl7nxCSEukAQgaq6kBI5vLp2duxaK8qBpBxg6H0KnQdP2ZkcohCiDJABRfalHjH79\n6OWQlQCu3sZY/fCZ0LafzM4VwsZJAhBVk38OYlcZ/foJO0E5GAuqjHgWuo0D50ZmRyiEqCRJAKJi\nxRZjyGbUEjj4HRQXgm8QjHgOevwFvFuZHaEQohpqlACUUi8DE4DzwFHgFq11RhnbHQeygWLAorWO\nqMlxRR05E1M6O3cF5KaAezOIuKV0dm6YdPEIUc/V9ApgA/CY1tqilPo38BjwSDnbDtVap9bweKK2\n5Zz9c3Zucgw4OEPXUUa/fucR4ORidoRCCCupUQLQWv9w0dPtwPU1C0eYwlJodO1ELzUKseliaN0T\nxrwM3aeCRzOzIxRC1AJr3gO4FVheznsa+EEppYH3tdaLrHhcUR1aQ+Iuo18/9ksoyACvVnDV3yFs\nJvh1MztCIUQtqzABKKU2Ai3LeGu+1npN6TbzAQuwuJzdDNRaJyml/IANSqkDWuvN5RxvLjAXoF27\ndpX4CqJKMhNK185dBmmHwcntz9m5HYfI7Nx6pKioiISEBAoKCswORZjAzc0Nf39/nJ2rXzRRaa1r\nFIRSajYwDximtc6rxPbPADla61cq2jYiIkJHRkbWKD4BnM81lk+MWgLHNgMa2l1lrKgVPBncvM2O\nUFTDsWPH8PLyolmzZii5IW9XtNakpaWRnZ1Nhw4d/uc9pdSuyg60qekooNEYN30Hl9f4K6U8AAet\ndXbp45HAczU5rqiEkhJjofTopcbC6edzoEl7GPwIhE0Hnw4V70PYtIKCAgICAqTxt0NKKZo1a0ZK\nSkqN9lPTewDvAK4Y3ToA27XW85RSrYEPtNZjgRbA6tL3nYAlWuvva3hcUZ60o0b3zt5lRjE2Fy8I\nmWz067cbAA4OZkcorEgaf/tljf/7mo4C6lzO60nA2NLH8UBYTY4jKpCfAXGrjbP9UzsABZ2GwrVP\nQrfx4OJudoSiAfL09CQnJ+d/Xnvttdf44IMPcHJywtfXl48++oj27S+vBVXWZ0Xdk5nA9VWxBeI3\nlc7OXQeWAmgeCMOfMQqwebc2O0Jhh3r27ElkZCTu7u689957PPzwwyxfXt7gQGE26Q+ob5L3wQ9P\nwOvBsPh6Iwn0vAn+9hPctQOuvl8af2GaoUOH4u5uXHH279+fhISEK26vteahhx6ie/fu9OjR40Ky\nOH36NIMGDSI8PJzu3buzZcsWiouLmTNnzoVtX3/99Vr/Pg2dXAHUB7mpELPSqLx5OhocnKDLSGPo\nZtdR4ORqdoTCZM9+Hce+pCyr7jO4tTdPTwip9uc//PBDxowZc8VtVq1aRVRUFNHR0aSmptKnTx8G\nDRrEkiVLGDVqFPPnz6e4uJi8vDyioqJITEwkNjYWgIyMy6rOiCqSBGCrLOfh8HqjJMPh9VBigZah\nMPolo+SyR3OzIxSiXF988QWRkZH88ssvV9xu69atzJgxA0dHR1q0aMHgwYPZuXMnffr04dZbb6Wo\nqIjJkycTHh5Ox44diY+P5+6772bcuHGMHDmyjr5NwyUJwJZoDUm7jUY/dqVRetmzBfS/wzjbb1H9\nszHRsNXkTN3aNm7cyAsvvMAvv/yCq+uVr07Lm4c0aNAgNm/ezLfffstNN93EQw89xM0330x0dDTr\n169n4cKFrFixgo8++qg2voLdkARgC7KSYO9yo+FPPQiOrkZt/fCZ0HEoOMp/k6gf9uzZw+233873\n33+Pn59fhdsPGjSI999/n9mzZ5Oens7mzZt5+eWXOXHiBG3atOFvf/sbubm57N69m7Fjx+Li4sLU\nqVPp1KkTc+bMqf0v1MBJy2KW83lw4FujXz/+Z9Alxipa49+AkOugUROzIxTiivLy8vD397/w/IEH\nHmDdunXk5ORwww03AEY5l7Vr15a7j+uuu45t27YRFhaGUooFCxbQsmVLPv30U15++WWcnZ3x9PTk\ns88+IzExkVtuuYWSkhIA/vWvf9XuF7QDNS4FUZsaXCkIreHkNmPoZtxXcD4bGrc1ZuaGzYBmncyO\nUNQj+/fvJygoyOwwhInK+huos1IQopLSj5UWYFsKGSfA2aN0du4MaD9QZucKIUwhCaC2FGTBvq+M\nfv2TvwEKOgyCoY8b1TddPMyOUAhh5yQBWFNJsdGfH70U9n8Dlnxo1tkoyRA6DZq0NTtCIYS4QBKA\nNaQcNPr1966A7CRwa2yUWg6bCf4RsnauEMImSQKorrz00tm5S42x+8oRuoyA0f+CrqPB2c3sCIUQ\n4ookAVRFcREc/sE42z+0HkqKoEUPGPWiMTvXs+Jxz0IIYStk+ElFtIakKPjuEXg1EJbNNEou950L\n87bCHVthwF3S+Au7k5yczMyZM+nYsSO9e/dmwIABrF69ukb7fOaZZ3jlFWOxwKeeeoqNGzdWaz9R\nUVGsW7euyp+bMWMGoaGhlxWauziuP5w6dYqhQ4cSFBRESEgIb775Zpn7LOuztkKuAMqTfcbo049e\nCmf3gaMLBI4x+vU7DwPH6q/DKUR9p7Vm8uTJzJ49myVLlgBw4sSJMid9WSwWnJyq3tQ891z1Fw6M\niooiMjKSsWPHVvozZ86c4bfffuPEiROV2t7JyYlXX32VXr16kZ2dTe/evRkxYgTBwcHVDbvOyRXA\nxYryjX79L66H14Jgw5PGcM1xr8GDB+Evn0HgaGn8hd376aefcHFxYd68eRdea9++PXfffTcAn3zy\nCTfccAMTJkxg5MiR5OTkMGzYMHr16kWPHj1Ys2bNhc+98MILBAYGMnz4cA4ePHjh9Tlz5rBy5UoA\ndu3axeDBg+nduzejRo3i9OnTAAwZMoRHHnmEvn370rVrV7Zs2cL58+d56qmnWL58OeHh4ZetR1BQ\nUMAtt9xCjx496NmzJ5s2bQJg5MiRnD17lvDwcLZs2VLh76BVq1b06tULAC8vL4KCgkhMTLziZ6Ki\noujfvz+hoaFcd911nDt3DoC33nqL4OBgQkNDmT59OgC//PIL4eHhhIeH07NnT7KzsyuMqarkCkBr\no0vnj9m5hZng7W/U1Q+bAc27mB2hEBX77lE4E2PdfbbsAWNeKvOtuLi4C41febZt28bevXvx8fHB\nYrGwevVqvL29SU1NpX///kycOJHdu3ezbNky9uzZg8VioVevXvTu3ft/9lNUVMTdd9/NmjVr8PX1\nZfny5cyfP/9CITiLxcLvv//OunXrePbZZ9m4cSPPPfcckZGRvPPOO5fFtXDhQgBiYmI4cOAAI0eO\n5NChQ6xdu5bx48cTFRVV5V/V8ePH2bNnD/369bvidjfffDNvv/02gwcP5qmnnuLZZ5/ljTfe4KWX\nXuLYsWO4urpeKHP9yiuvsHDhQgYOHEhOTg5ubtYfWGK/CeDcCaMAW/RSSI8HZ3cImmgM3wwYJLNz\nhaiCu+66i61bt+Li4sLOnTsBGDFiBD4+PoDRZfT444+zefNmHBwcSExMJDk5mS1btnDdddddWERm\n4sSJl+374MGDxMbGMmLECACKi4tp1arVhfenTJkCQO/evTl+/HiFsW7duvXClUq3bt1o3749hw4d\nwtvbu1rfPScnh6lTp/LGG29ccR+ZmZlkZGQwePBgAGbPnn2hZlJoaCizZs1i8uTJTJ48GYCBAwfy\nwAMPMGvWLKZMmfI/dZespUYJQCn1DPA34I+l6R/XWl9250UpNRp4E3DEWCy+7NOK2laYDfvWGo3+\n8dJLvIBr4Jp/QPBEcPUyJSwhaqycM/XaEhISwpdffnnh+cKFC0lNTSUi4s8SNB4ef852X7x4MSkp\nKezatQtnZ2cCAgIoKCgAKl7cXGtNSEgI27ZtK/P9P0pOOzo6YrFYKozdmvXPioqKmDp16oVGurq+\n/fZbNm/ezNq1a3n++eeJi4vj0UcfZdy4caxbt47+/fuzceNGunXrZrXYwTr3AF7XWoeX/iur8XcE\nFgJjgGBghlKq7u6SlBTD0U2w6nZ4pSusuROyEmHoE3DvXpjzDfScJY2/EFVw7bXXUlBQwHvvvXfh\ntby8vHK3z8zMxM/PD2dnZzZt2nThRuugQYNYvXo1+fn5ZGdn8/XXX1/22cDAQFJSUi4kgKKiIuLi\n4q4Yn5eXV7l95oMGDWLx4sUAHDp0iJMnTxIYGHjlL1wGrTW33XYbQUFBPPDAAxVu37hxY5o2bXrh\n/sLnn3/O4MGDKSkpuTCiaMGCBWRkZJCTk8PRo0fp0aMHjzzyCBERERw4cKDKMVakLrqA+gJHtNbx\nAEqpZcAkYF+tHjX1cOns3OVGg+/aGEL/YoziadtXZucKUQNKKb766ivuv/9+FixYgK+vLx4eHvz7\n3/8uc/tZs2YxYcIEIiIiCA8Pv3Am26tXL6ZNm0Z4eDjt27fnmmuuueyzLi4urFy5knvuuYfMzEws\nFgv33XcfISHlL4IzdOhQXnrpJcLDw3nssceYNm3ahffuvPNO5s2bR48ePXBycuKTTz6pcOEagH/+\n85+88cYbF54vW7aMzz//nB49ehAeHg7Aiy++eMWRR59++inz5s0jLy+Pjh078vHHH1NcXMyNN95I\nZmYmWmvuv/9+mjRpwpNPPsmmTZtwdHQkODi4wuU1q6NG5aBLu4DmAFlAJPCg1vrcJdtcD4zWWv+1\n9PlNQD+t9d8r2n+1ykEX5sBnkyAxEpQDdBpm9OsHjgXnRlXblxA2TMpBi1ovB62U2gi0LOOt+cB7\nwPOALv35KnDrpbso47PlZh2l1FxgLhiLSVSZq6dRVz94knHG71VW6EIIISpMAFrr4ZXZkVLqv8A3\nZbyVAFxcBtMfSLrC8RYBi8C4AqjMsS8zZVG1PiaEEPakRjeBlVKtLnp6HRBbxmY7gS5KqQ5KKRdg\nOlD+GnFCCCHqRE1vAi9QSoVjdOkcB24HUEq1xhjuOVZrbVFK/R1YjzEM9COt9ZVv4QshKkVrXeEw\nStEwWWM4a40SgNb6pnJeTwLGXvR8HVD1ykxCiHK5ubmRlpZGs2bNJAnYGa01aWlpNZ4dbL8zgYWo\n5/z9/UlISCAlJaXijUWD4+bmVuPZwZIAhKinnJ2d6dChg9lhiHpMCt4IIYSdkgQghBB2ShKAEELY\nqRqVgqhtSqkUoHLL89iO5kCq2UHUMfnO9kG+c/3QXmvtW5kNbToB1EdKqcjK1uFoKOQ72wf5zg2P\ndAEJIYSdkgQghBB2ShKA9dljJTr5zvZBvnMDI/cAhBDCTskVgBBC2ClJALVIKfUPpZRWSjU3O5ba\nppR6WSl1QCm1Vym1WinVxOyYaotSarRS6qBS6ohS6lGz46ltSqm2SqlNSqn9Sqk4pdS9ZsdUF5RS\njkqpPUqpstY5aRAkAdQSpVRbYARw0uxY6sgGoLvWOhQ4BDxmcjy1QinlCCwExgDBwAylVLC5UdU6\nC8Zyr0FAf+AuO/jOAPcC+80OojZJAqg9rwMPc4XlLxsSrfUPWmtL6dPtGCu/NUR9gSNa63it9Xlg\nGTDJ5Jhqldb6tNZ6d+njbIxGsY25UdUupZQ/MA74wOxYapMkgFqglJoIJGqto82OxSS3At+ZHUQt\naQOcuuh5Ag28MbyYUioA6AnsMDeSWvcGxglcidmB1CYpB11NSqmNQFkrzs8HHgdG1m1Ete9K31lr\nvaZ0m/kYXQaL6zK2OlTWyit2cZWnlPIEvgTu01pnmR1PbVFKjQfOaq13KaWGmB1PbZIEUE1a6+Fl\nva6U6gF0AKJLV2nyB3Yrpfpqrc/UYYhWV953/oNSajYwHhimG+744gSg7UXP/YEkk2KpM0opZ4zG\nf7HWepXZ8dSygcBEpdRYwA3wVkp9obW+0eS4rE7mAdQypdRxIEJrXd8KSlWJUmo08BowWGvdYJeo\nUko5YdzkHgYkAjuBmQ15nWtlnMl8CqRrre8zO566VHoF8A+t9XizY6kNcg9AWMs7gBewQSkVpZT6\nj9kB1YbSG91/B9Zj3Axd0ZAb/1IDgZuAa0v/b6NKz45FPSdXAEIIYafkCkAIIeyUJAAhhLBTkgCE\nEMJOSQIQQgg7JQlACCHslCQAIYSwU5IAhBDCTkkCEEIIO/X/lGfpVZhqZisAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10d2c46a0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = gluon.loss.L2Loss()\n",
    "\n",
    "with ag.record():    # start recording\n",
    "    theloss = loss(output, thelabel)\n",
    "theloss.backward()   # and compute the gradient\n",
    "\n",
    "plt.plot(output.asnumpy(), theloss.asnumpy(), label='L2 loss')\n",
    "plt.plot(output.asnumpy(), output.grad.asnumpy(), label='Gradient of L2 loss')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Huber's Robust loss\n",
    "\n",
    "Huber's Robust Loss is a cross between the L1 and the L2 loss. It behaves like an L2 loss close to zero. Beyond that, for discrepancies larger than $\\rho$ it behaves like an L1 loss. The scaling is set up in such a way as to ensure that the derivative is continuous. \n",
    "\n",
    "$$l(y,f) = \\begin{cases}\n",
    "\\frac{1}{2 \\rho} (y-f)^2 & \\text{ for } |y-f| < \\rho \\\\\n",
    "|y-f| - \\frac{\\rho}{2} & \\text{ otherwise}\n",
    "\\end{cases}$$\n",
    "\n",
    "If we minimize the loss something interesting happens (again, we're in the toy scenario that we just estimate a scalar). The number of cases with $y_i < f-\\rho$ and with $y_i > f+\\rho$ are going to cancel out, since their gradients are all $-1$ and $1$ respectively. For all the $y_i$ closer to $f$, the gradients will balance out like in the L2 loss case. In other words, $f$ will be the mean for all points closer than $\\rho$. This is pretty much what a *trimmed mean* estimator does. It ensures that a few outliers (very large $|y_i|$) won't break the estimate. Let's check it out in practice."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xtcjvf/wPHX1VkkoZxCmsV0kE4iyiGHfZkfs83MoeRU\niLUxp2lzmuOclfk61MawDWObbeSUMJ0UOTVLKKKDIul0d/3+8NV2LTne3JXP8/HY4+vufd2f633f\n37xd93V/Pu+PJMsygiAIQtWhpekEBEEQBPUShV0QBKGKEYVdEAShihGFXRAEoYoRhV0QBKGKEYVd\nEAShihGFXRAEoYrRUccgkiQlA3cAFVAsy7KTOsYVBEEQnp5aCvv/dJZlOUON4wmCIAjPQJ2F/YnV\nrVtXtrCw0MSpBUEQKq2YmJgMWZZNH3ecugq7DOyVJEkGvpJlee2jDrawsCA6OlpNpxYEQXg1SJJ0\n+UmOU1dhd5Nl+ZokSWbAPkmSzsuyHP6vhEYBowCaNGmiptMKgiAI/6aWWTGyLF/73//eBHYCLg85\nZq0sy06yLDuZmj72k4QgCILwjJ67sEuSVF2SJKMHfwa6AwnPO64gCILwbNRxK6YesFOSpAfjfSvL\n8m9qGFd4QkVFRaSkpJCfn6/pVAShlIGBAebm5ujq6mo6lVfOcxd2WZaTgNZqyEV4RikpKRgZGWFh\nYcH//oEVBI2SZZnMzExSUlJo1qyZptN55YiVp1VAfn4+derUEUVdqDAkSaJOnTriU6SGiMJeRYii\nLlQ04ndScypVYT955RZrDv+l6TQEQRCeSczlrJdynkpV2HeeTGX+r+fZfOKJ5ugLL1GNGjUUj0NC\nQhg3btwjn3Po0CF69+6tthw+//xzFi9erLbx/um3336jRYsWNG/enPnz5z/0mJCQEExNTbG3t8fe\n3p5169a9kFyEymnzicv0Dz7Oz6euvfBzaaSlwLMK7N2Kq1l5zPgxgXpGBni2qqfplAQNUqlUaGtr\nv5TzjB07ln379mFubo6zszN9+vShVatWZY4dMGAAq1ateuE5CZXL/nM3mPFjAp1bmNLTuv4LP1+l\numLX0dZi1QcO2DQyZtyWWE5euaXplIQn4O3tzQ8//FD6+J9X97dv36Zfv360atUKX19fSkpKANi7\ndy/t2rXDwcGBd999l9zcXOB+O4pZs2bRoUMHvv/++3LPGRcXh6urK3Z2dvTr149bt+7/rqxYsYJW\nrVphZ2fH+++/D8Dhw4dLr7LbtGnDnTt3FGNFRkbSvHlzLC0t0dPT4/3332fXrl3qeXOEKi/uajbj\nvj2JdUNjVn3ggI72iy+7leqKHaC6vg4bvJ15O+gYw0Oj2e7XnmZ1q2s6rQpj5k9nOHvttlrHbNWw\nJp+9Zf3IY+7du4e9vX3p46ysLPr06fPYsSMjIzl79ixNmzalZ8+e7Nixg06dOjFnzhzCwsKoXr06\nCxYsYMmSJQQGBgL350dHREQ8ctyhQ4eycuVKPDw8CAwMZObMmSxbtoz58+dz6dIl9PX1yc7OBmDx\n4sWsXr0aNzc3cnNzMTAwUIyVmppK48aNSx+bm5tz4sSJh553+/bthIeHY2VlxdKlSxXPE149yRl3\n8QmJwtRInw3ezlTXfzklt1JdsT9Qt4Y+oT73uxZ4b4wkI7dAwxkJ1apVIy4urvS/WbNmPdHzXFxc\nsLS0RFtbm4EDBxIREcEff/zB2bNncXNzw97entDQUC5f/vt7lQEDBjxyzJycHLKzs/Hw8ADAy8uL\n8PD7rYvs7OwYNGgQmzZtQkfn/l8yNzc3PvroI1asWEF2dnbpzx+QZbnMOR424+Ott94iOTmZU6dO\n4enpiZeX1xO9B0LVlJFbgNfGSGRZJmSYM6ZG+i/t3JXuiv2BZnWrs97LiYH//YPhIVFsGeWKoV6l\nfTlq87gra03Q0dEpvcUiyzKFhYWlsX8XSEmSkGWZbt26sWXLloeOV736s39C++WXXwgPD2f37t3M\nnj2bM2fOMGXKFHr16sWePXtwdXUlLCyMli1blj7H3Nycq1evlj5OSUmhYcOGZcauU6dO6Z9HjhzJ\n5MmTnzlPoXLLKyxmeEgUN27n8+1IVyxNazz+SWpUKa/YH2jTxIRVAx04nZrD2M2xFKtKNJ2S8BAW\nFhbExMQAsGvXLoqKikpjkZGRXLp0iZKSErZt20aHDh1wdXXl6NGjXLx4EYC8vDwSExOf+HzGxsaY\nmJhw5MgRAL755hs8PDwoKSnh6tWrdO7cmYULF5KdnU1ubi5//fUXtra2TJ48GScnJ86fP68Yz9nZ\nmT///JNLly5RWFjI1q1bH3qb6fr166V/3r17N2+88caTv0lClVGsKmHctyc5nZrDyoEOODQxeek5\nVKpLXFmWySvOo7ru31dsnq3qMbuvDdN3JvDpjwnMe9tWLIyoYEaOHMn//d//4eLiQteuXRVX3O3a\ntWPKlCmcPn0ad3d3+vXrh5aWFiEhIQwcOJCCgvu32ebMmYOVldUTnzM0NBRfX1/y8vKwtLRk48aN\nqFQqBg8eTE5ODrIsExAQQK1atZgxYwYHDx5EW1ubVq1a8eabbyrG0tHRYdWqVfTo0QOVSoWPjw/W\n1vc/GQUGBuLk5ESfPn1YsWIFu3fvRkdHh9q1axMSEvL8b55QqciyzIxdCRw4f5M5fW3opqGZe9LD\n7h++aE5OTvKzbLSxNGYpEakRhPYMpYae8qPN4t8vsOrgRQI8rZjg+bq6Uq0Uzp07J64OhQrpVfvd\nXLH/T5bsS2Rs59eY1KOlIpZbmMu8yHkEOAZQt1rdZxpfkqSYJ9lTulLdimnboC1J2Ul8eOhDilRF\nitjH3a3o72DO0rBEvou6Ws4IgiAIL8b30VdZsi+Rtx0aMbF7C0WsSFXEh4c+ZE/SHv7KfvGr5ytV\nYW/fsD0z3WZy4voJZhybQYn89z11SZKY39+Wjq/XZerO0xy8cFODmQqC8Co5nJjO1B2n6dC8LvPf\ntlPcDpZlmcBjgZy4foKZbjNp26DtC8+nUhV2gD6v9WF8m/H8kvQLy2OXK2K62loED3akZX0jxm6O\n5VRKtoayFAThVZGQmoPfphis6hkRPNgBPR1lWV0eu5yfk35mfJvx9Hnt8Ws71KHSFXaAEbYjeM/q\nPTYkbGDLeeWUuBr6Omz0dsbEUA+fkCiuZOZpKEtBEKq6q1l5eG+MwsRQj43DnDEyUG4qsvX8VtYn\nrOc9q/cYYTvipeVVKQu7JElMazuNTo07Me/EPPZf3q+Im9U0INTHheISGa+NkWSKBUyCIKjZrbuF\neG2MpEhVQqiPM/VqKlcs77+yny9OfEEn805MbTv1pc7Wq5SFHUBbS5uF7guxNbVl8pHJxN2MU8Sb\nm9Vg3VAnrmXfY3hoNPcKVRrKVBCEqia/SMXw0ChSbt1jnZcTzc2MFPG4m3FMDp+MbV1bFnosREfr\n5c4sr7SFHaCaTjVWdVlF/er1GXdgHJdyLiniTha1Wf5+G+JTsvHfclIsYHqBbty4wQcffIClpSWO\njo60a9eOnTt3PteY/2zDGxgYSFhY2DONExcXx549e576eQMHDsTOzo6lS5eWm9cDFhYWZGRkPHK8\nTp068SzTfMvz71bJ6lJQUMCAAQNo3rw5bdu2JTk5+aHHWVhYYGtri729PU5Oj52BV2WoSmTGbznJ\nyavZLB9gj7NFbUU8OScZ/wP+1DOsx8quK6mmU+2l51ipCzuAiYEJwZ7BaEva+IX5kXFP+Zerp019\nZvaxJuzcDT7bfeahfT+E5yPLMn379sXd3Z2kpCRiYmLYunUrKSkpZY4tLi5+pnPMmjULT0/PZ3ru\nsxT2tLQ0jh07xqlTpwgICHim86rTs75vz2L9+vWYmJhw8eJFAgICHtka4eDBg8TFxan1H6yKTJZl\nZv50hr1nbxDYuxVv2jZQxDPuZeAb5ouWpMUazzXUNqhdzkgvVqUv7ACNjRoT5BlEVn4WY8LGcLfo\nriI+tJ0Fvh6vsfnEFYIOiR2Y1O3AgQPo6enh6+tb+rOmTZvi7+8P3N+A4t133+Wtt96ie/fu5Obm\n0rVrVxwcHLC1tVW0wJ07dy4tWrTA09OTCxculP78n61/Y2Ji8PDwwNHRkR49epQu5e/UqROTJ0/G\nxcUFKysrjhw5QmFhIYGBgWzbtg17e3u2bdumyD0/P59hw4Zha2tLmzZtOHjwIADdu3fn5s2b2Nvb\nl7YmeBLJycnY2NiUPl68eDGff/556eNNmzbRvn17bGxsiIyMBODu3bv4+Pjg7OxMmzZtSt+Pf79v\n5ZFlmUmTJmFjY4OtrW3pa7x+/Tru7u7Y29tjY2PDkSNHUKlUeHt7lx77708jcL/tw4MGZu+88w77\n9+8XF0T/81V4El8fv8wod0uGuSk36c4rymPs/rFk5WexuutqGtfUXGfPStVS4FGs61jzpceX+B/w\n5+NDH7Oy60p0tf7+hvqTHi1Iy7nHot8vUL+mAf0dzTWY7Qv06xRIO63eMevbwpsP3zUI4MyZMzg4\nODxyiOPHj3Pq1Clq165NcXExO3fupGbNmmRkZODq6kqfPn2IjY1l69atnDx5kuLiYhwcHHB0dFSM\nU1RUhL+/P7t27cLU1JRt27Yxffp0NmzYANy/so2MjGTPnj3MnDmTsLAwZs2aRXR09EM3wFi9ejUA\np0+f5vz583Tv3p3ExER2795N7969iYuLK/McgKVLl7Jp06bSx9euPdmuOHfv3uXYsWOEh4fj4+ND\nQkICc+fOpUuXLmzYsIHs7GxcXFxKP538830rz44dO4iLiyM+Pp6MjAycnZ1xd3fn22+/pUePHkyf\nPh2VSkVeXh5xcXGkpqaSkJAAUNq6+J/+2aZYR0cHY2NjMjMzqVtXuVpSkiS6d++OJEmMHj2aUaNG\nPdF7UFntiru/g9tbrRsypadyVWlRSREfHf6IC1kXWNFlBTZ1bcoZ5eWoMoUdoKN5Rz5r9xmBxwL5\n/NjnzHGbU/pNtJaWxMJ3WpOeW8Dk7acwNdLH3cpUwxlXTWPHjiUiIgI9PT2ioqIA6NatW2lxkmWZ\nadOmER4ejpaWFqmpqdy4cYMjR47Qr18/DA0NAR7aaOvChQskJCTQrVs34P7uRg0a/P1x+O233wbA\n0dGx3HvD/xQREVH6yaJly5Y0bdqUxMREatas+cjnBQQEMHHixNLHFhYWjz0X3L9vD+Du7s7t27fJ\nzs5m79697N69u/S+fX5+PleuXAGU79ujXsPAgQPR1tamXr16eHh4EBUVhbOzMz4+PhQVFdG3b1/s\n7e2xtLQkKSkJf39/evXq9dBPAk/apvjo0aM0bNiQmzdv0q1bN1q2bIm7u/sTvQ+VzdGLGUz8Ph5X\ny9osftcOLS3lAqRZx2dxNPUoM9vPxN1c8+9BlSrsAP1e70daXhpBcUHUr14f/zb+pTE9nfsLmN5b\ncxy/TTFsG90Om0bGGsz2BXjElfWLYm1tzfbt20sfr169moyMDMUXav9s/LV582bS09OJiYlBV1cX\nCwsL8vPzgcfvbC/LMtbW1hw/fvyhcX39+z2vtbW1n+i+tLpvMfyzRTFQ+roeKK9N8fbt22nRQrkM\n/cSJE0/Uori81+Du7k54eDi//PILQ4YMYdKkSQwdOpT4+Hh+//13Vq9ezXfffVf6aeeBB22Kzc3N\nKS4uJicn56H/uDxoXWxmZka/fv2IjIyskoX93PXb+H4Tg2XdGnw1xAl9HeV2jEHxQfx48Uf8Wvvx\n9utvayhLpSpxj/3ffO186f96f9aeWst3F75TxGoa6BLq44JxNV2GhURxNUssYHpeXbp0IT8/n+Dg\n4NKf5eWV/77m5ORgZmaGrq4uBw8eLN1Ew93dnZ07d3Lv3j3u3LnDTz/9VOa5LVq0ID09vbSwFxUV\ncebMmUfmZ2RkVGa7uwfc3d3ZvHkzAImJiVy5cqVMgX0a9erV4+bNm2RmZlJQUMDPP/+siD+4/x0R\nEYGxsTHGxsb06NGDlStXlhbokydPPtU53d3d2bZtGyqVivT0dMLDw3FxceHy5cuYmZkxcuRIhg8f\nTmxsLBkZGZSUlNC/f39mz55NbGxsmfH69OlDaGgoAD/88ANdunQp8w/S3bt3S9/Tu3fvsnfvXsV3\nC1VFavY9vDdGUl1fh43DnDGuplyA9EPiD6yJX0O/5v3wa+2noSzLUlthlyRJW5Kkk5Ik/fz4o18s\nSZL41PVT3M3dmXtiLgevHFTE6/1vAVNBkQrvjZFk5xWWM5LwJCRJ4scff+Tw4cM0a9YMFxcXvLy8\nWLBgwUOPHzRoENHR0Tg5ObF58+bSTS0cHBwYMGAA9vb29O/fn44dO5Z5rp6eHj/88AOTJ0+mdevW\n2Nvbc+zYsUfm17lzZ86ePfvQL0/HjBmDSqXC1taWAQMGEBISUnrV/yx0dXUJDAykbdu29O7dW7Fh\nB4CJiQnt27fH19eX9evXAzBjxgyKioqws7PDxsaGGTNmPNU5+/Xrh52dHa1bt6ZLly4sXLiQ+vXr\nc+jQodJ9XLdv386ECRNITU2lU6dO2Nvb4+3tzbx588qMN3z4cDIzM2nevDlLlixh/vz7nwKvXbvG\nf/7zH+D+9NYOHTrQunVrXFxc6NWrFz179nyWt6zCyskrwntDJHkFKkJ8nGlYSzltMTwlnDl/zMGt\nkRsz2s2oUO3C1da2V5KkjwAnoKYsy70fdeyztu19WnlFeQz/fTgXsy+yrsc6Wpu2VsRPJGUyZH0k\ndubGbBrRFgPdF7/j/YvwqrVGFSqPyvq7mV+kYuiGSOKuZBPi40z715RfHCdkJODzuw/NjJuxscdG\nDHUNX0peL7VtryRJ5kAvYJ06xlMXQ11DVnVdhamhKf77/bl8+7Ii3tayDksH2BNz5RYfbo1DVSKm\ndAnCq66kRObj7+OJvJTF4vdalynqV29fZez+sdQ2qM3qrqtfWlF/Guq6FbMM+ASocEs761SrwxrP\nNUiShO8+3zILmHrZNeDTXq347Uwas38+K+brCsIr7os95/jl1HWm/aclfVor97bNys/CN8yXErmE\nNZ5rnnnDjBftuQu7JEm9gZuyLMc85rhRkiRFS5IUnZ6e/rynfSpNajZhVZdVZOZnMm7/OPKKlF/s\nDe/QjBEdmhFyLJm14UkvNTdBECqO9RGXWBdxCe/2FozsaKmI3Su+h/9+f27k3WBll5VYGFtoJskn\noI4rdjegjyRJycBWoIskSZv+fZAsy2tlWXaSZdnJ1PTlzx+3NbVlkfsizmWdY+LhiRSXKKfCTfvP\nG/S2a8C8X8+zKy71pecnCIJm/XLqOnN+OUtP6/rM6N1K8WVocUkxnxz+hITMBBa6L8TezF6DmT7e\ncxd2WZanyrJsLsuyBfA+cECW5cHPndkL4NHYgxmuMziSeoTZf8xW3HbR0pL48r3WtG1Wm4nfx3Ps\n4qMbOgmCUHWcSMokYFscjk1MWPa+Pdr/WoD0xYkvOJRyiGku0+jSpIsGM30yVXIe+6O8Y/UOo+1G\ns+PPHayJX6OI6etos3aoE83qVmf0NzGcu35bQ1kKgvCyJN64w8ivo2lcuxrrvJzKzI777+n/8n3i\n9wy3Gc6AlgM0lOXTUWthl2X50OOmOlYEY+3H0rd5X4Lig9ieuF0RM66mS8gwF6rr6+C9MZLU7Hsa\nyrJy+XcL2ZCQEMaNG/fI5xw6dIjevdX36/Kwdrrq4uPjg5mZWZVchPMqS8vJx3tDJPq62oQMc6GW\noZ4ivuviLlaeXElvy95McJigoSyf3it3xQ73F9QEtgvErZEbs/+YTXhKuCLesFY1QnycyStQ4b0h\nkpy8Ig1lKjyKSvXyNk/x9vbmt99+e2nnE1682/lFeG+MJOdeESHDnGlcWzlt8VjqMT4/9jmuDVyZ\n1X5WhVqA9DivZGEH0NXSZYnHEqxMrJh4eCIJGQmKeMv6NflqqCPJmXcZ+U00+UViB6Zn9c+Wu6C8\nur99+zb9+vWjVatW+Pr6lvZZ2bt3L+3atcPBwYF3332X3Nxc4H6zrVmzZtGhQwe+//77cs8ZFxeH\nq6srdnZ29OvXj1u3bgGwYsUKWrVqhZ2dHe+//z4Ahw8fxt7evnSV5sPaD7i7uz+2GZdQeRQWl+C3\nKYaLN3NZM8QR64bKnlHnMs8RcCiA12q9xtJOS9HV1i1npIqpyjUBexqGuoYEeQYxeM9gxu4fy6Y3\nNyl6KLd/rS5fvmfP+C0n+fj7eFa+30bR1a0iWhC5gPNZ59U6ZsvaLZnsUv5mCwD37t3D3v7vmQJZ\nWVkP7c74b5GRkZw9e5amTZvSs2dPduzYQadOnZgzZw5hYWFUr16dBQsWsGTJEgIDAwEwMDAgIiLi\nkeMOHTqUlStX4uHhQWBgIDNnzmTZsmXMnz+fS5cuoa+vX9qydvHixaxevRo3Nzdyc3MxMDB45NhC\n5SbLMp/8EM/Ri5l8+W5rOr6unKWXmpvKmP1jMNY3JsgziBp6L2anqhfplb1if6ButboEewZTIpfg\nG+ZLVn6WIt6ndUOm/aclv5y6ztw95zSUZcVXrVo14uLiSv+bNWvWEz3PxcUFS0tLtLW1GThwIBER\nEfzxxx+cPXsWNzc37O3tCQ0NLW0UBjBgwKO/wMrJySE7OxsPDw8AvLy8CA+/f7vNzs6OQYMGsWnT\nJnR07l/XuLm58dFHH7FixQqys7NLfy5UTQt/v8CPcdeY2N2qzL4MOQU5+IX5UaAqINgzGDNDMw1l\n+XzEbzDQzLgZK7usZMTeEfjv92ddj3WKfQpHdrTkWnY+6yMu0cDYgBH/WrhQkTzuyloT/tnKVpZl\nCgv/brpWXhvbbt26sWXLloeO9yStbMvzyy+/EB4ezu7du5k9ezZnzpxhypQp9OrViz179uDq6kpY\nWFiZ5l1C1fDN8WSCD/3FB22bMLZzc0Usvzgf/wP+pN5JZW33tbxW6zXNJKkGr/wV+wP2ZvYsdF9I\nQmYCnxz+RLGASZIkZvRuxZs29Znzyzl+PvVku+UI91lYWBATc39h8q5duygq+vvL6MjISC5dukRJ\nSQnbtm2jQ4cOuLq6cvToUS5evAjcbwGcmJj4xOczNjbGxMSkdEu7b775Bg8PD0pKSrh69SqdO3dm\n4cKFZGdnk5uby19//YWtrS2TJ0/GycmJ8+fVeytLqBh+P5NG4O4zeL5hxqw+1oqLClWJiqlHphJ3\nM455HefhWM/xESNVfKKw/0OXJl2Y5jKNQymH+OLEF4oFTNpaEksH2ONsYcJH2+L5IylTg5lWLiNH\njuTw4cO4uLiU2TyiXbt2TJkyBRsbG5o1a0a/fv0wNTUlJCSEgQMHYmdnh6ur61MX29DQUCZNmoSd\nnR1xcXEEBgaiUqkYPHhw6f6mAQEB1KpVi2XLlmFjY0Pr1q2pVq0ab775ZpnxBg4cSLt27bhw4QLm\n5ualLXeFyiHmchbjt5yktXktVg50QEf779InyzILoxYSdiWMSc6T6G5R/v6ylYXa2vY+jZfVtvdZ\nLY9dzrrT6/Bv488oO+U+jtl5hbyz5jg3b+fzg197rOoZaSjLv1XW1qhC1VcRfjf/Ss+lf/AxalXT\nZbtfe+rUUPbb35iwkSUxSxjaaiiTnCdpKMsn81Lb9lY149uM5y3Lt1h5ciU/XvxREatlqEfIMGcM\ndLXx2hBJWk5+OaMIgqBpN+/k47UhEh0tiVAflzJFfU/SHpbELKGnRU8+dvpYQ1mqnyjsDyFJEjPb\nz8S1gSszj83kaOpRRdzcxJCNw5y5k1+M98ZIbueLBUyCUNHkFhTjExJFZm4h672caVpH+aV75PVI\nph+djlM9J+Z2mIuWVHXKYdV5JWqmq63L0k5LaW7SnIBDAZzNPKuIWzc0JniwAxdv5uL7TQyFxRWu\nFb0gvLKKVCWM3RzLuet3CBrkQOvGtRTxxFuJTDg4AYuaFizvshw9bb1yRqqcRGF/hBp6NVjddTW1\n9GsxJmwMKXdSFPGOr5uy8B07jv2VyaQf4ikROzAJgsbJssy0Hac5nJjO3L42dG6pnIuedjcNvzA/\nDHUNCfYMpqZeTQ1l+uKIwv4YZoZmrPFcQ1FJEX5hfmTnZyvibzuYM6lHC3bFXWPB72KanCBo2tKw\nP/k+JoXxXV/nfZcmitjtwtv4hfmRV5RHsGcw9avX11CWL5Yo7E/AspYlK7us5FruNcYdGEd+sfIL\n0zGdXmOwaxO+OpxE6LFkzSQpCAJbIq+wYv+fvOdkToDn64pYoaqQCQcmkHw7mWWdl2FlYqWhLF88\nUdifkEM9B+a7z+dU+ikmh09GVfJ3UzBJkpjZx4Zurerx+U9n+C3hugYz1YwbN27wwQcfYGlpiaOj\nI+3atWPnzp3PNeY/2/AGBgYSFhb2TOPExcWxZ8+ep37eg3n0S5cuLTevBywsLMjIePTmLJ06dUKd\n03z/3SpZXcLDw3FwcEBHR0fRvK2iO3D+Bp/+mICHlSlz+9kqFiCVyCVMj5hO9I1o5rjNoW2DthrM\n9MUThf0pdGvajckukzlw9QDzI+eXWcC04v022DeuxfitcUQlZz1ipKpFlmX69u2Lu7s7SUlJxMTE\nsHXrVlJSUsocW1xc/JARHm/WrFl4eno+03OfpbCnpaVx7NgxTp06RUBAwDOdV52e9X17Fk2aNCEk\nJIQPPvjgpZ3zecVfzWbs5pO0alCToEEO6GorS9uS6CX8lvwbAY4B9LLspaEsXx5R2J/SoDcGMcx6\nGFsvbGVDwgZFrJqeNuu9nGlUqxojQqO5eLNs+9eq6MCBA+jp6eHr61v6s6ZNm+Lv7w/c33Tj3Xff\n5a233qJ79+7k5ubStWtXHBwcsLW1ZdeuXaXPmzt3Li1atMDT05MLFy6U/vyfrX9jYmLw8PDA0dGR\nHj16cP36/U9InTp1YvLkybi4uGBlZcWRI0coLCwkMDCQbdu2YW9vz7Zt2xS55+fnM2zYsNLVqAcP\nHgSge/fu3Lx5E3t7+9LWBE8iOTlZsRnH4sWL+fzzz0sfb9q0ifbt22NjY0NkZCQAd+/excfHB2dn\nZ9q0aVP6fvz7fSuPLMtMmjQJGxsbbG1tS1/j9evXcXd3x97eHhsbG44cOYJKpcLb27v02H9/GoH7\nnz7s7Oy73TnHAAAgAElEQVTQ0qoc5SE54y4+IVHUNdJjg7cz1fWVLbA2nd1E6NlQBrYcyDDrYRrK\n8uUSTcCewYeOH5KWl8ay2GWYGZrx1mtvlcZqV9cjdJgLbwcfxWtDFDvHtMes5strA5v2xRcUnFPv\nl7j6b7Sk/rRp5cbPnDmDg4PDI8c4fvw4p06donbt2hQXF7Nz505q1qxJRkYGrq6u9OnTh9jYWLZu\n3crJkycpLi7GwcEBR0dlz46ioiL8/f3ZtWsXpqambNu2jenTp7Nhw/1/ZIuLi4mMjGTPnj3MnDmT\nsLAwZs2aRXR0NKtWrSqT1+rVqwE4ffo058+fp3v37iQmJrJ792569+5NXFzcQ1/P0qVL2bTp7z3b\nr117sv5Bd+/e5dixY4SHh+Pj40NCQgJz586lS5cubNiwgezsbFxcXEo/nfzzfSvPjh07iIuLIz4+\nnoyMDJydnXF3d+fbb7+lR48eTJ8+HZVKRV5eHnFxcaSmppKQcH//gQetiyurzNwCvDdGUiLLhA5z\nwdRIuQBpb/JeFkYtpGuTrkx2nlypNst4HqKwPwMtSYs5bnPIvJdJ4NFA6larS7uG7UrjTeoYstHb\nhQFrj+O9MYpto10xMqhcjfqfx9ixY4mIiEBPT4+oqCgAunXrVlqcZFlm2rRphIeHo6WlRWpqKjdu\n3ODIkSP069cPQ8P7O9k8rJ/7hQsXSEhIoFu3bsD9XZQaNGhQGn/77bcBcHR0JDk5+bG5RkRElH6y\naNmyJU2bNiUxMZGaNR89BS4gIICJEyeWPrawsHjsueD+fXu4v3HH7du3yc7OZu/evezevbv0vn1+\nfj5XrlwBlO/bo17DwIED0dbWpl69enh4eBAVFYWzszM+Pj4UFRXRt29f7O3tsbS0JCkpCX9/f3r1\n6vXITwIV3b1CFT6h0VzPyefbka5Ymiq/c4i5EcPUI1OxN7Nnfsf5aGtplzNS1SMK+zPS09ZjWedl\neP3mRcChAEJ6htCy9t+tXm3NjQka5MDw0GjGbI5lvZczejov/qPto66sXxRra2u2b/9779jVq1eT\nkZGBk9PfLS3+2fhr8+bNpKenExMTg66uLhYWFuTn359p9LgrKlmWsba25vjx4w+N6+vfv2LT1tZ+\novvS6u6V9M8WxUDp63qgvDbF27dvp0WLForYvxumlae81+Du7k54eDi//PILQ4YMYdKkSQwdOpT4\n+Hh+//13Vq9ezXfffVf6aacyKVaV4L8lltMp2QQPdsSxqYkinpSdxPgD42lYoyErOq/AQOfV2jyl\nctxEq6CM9IwI6hpEDd0ajAkbw7Vc5cfxTi3MmP+2LUf+zGDKjlNqLyIVRZcuXcjPzyc4OLj0Z3l5\neeUen5OTg5mZGbq6uhw8eLB0Ew13d3d27tzJvXv3uHPnDj/99FOZ57Zo0YL09PTSwl5UVMSZM2ce\nmZ+RkdFDt7t7cM7NmzcDkJiYyJUrV8oU2KdRr149bt68SWZmJgUFBfz888+K+IP73xERERgbG2Ns\nbEyPHj1YuXJl6e/HyZMnn+qc7u7ubNu2DZVKRXp6OuHh4bi4uHD58mXMzMwYOXIkw4cPJzY2loyM\nDEpKSujfvz+zZ88mNjb2mV+rpsiyTODuM4Sdu8nMPtb0sFbORb+ZdxPfMF/0tPVY020NtQxqlTNS\n1SUK+3OqX70+azzXkF+cj1+YHzkFOYr4u06N+aibFTtiU1m890I5o1RukiTx448/cvjwYZo1a4aL\niwteXl4sWLDgoccPGjSI6OhonJyc2Lx5c+mmFg4ODgwYMAB7e3v69+9Px44dyzxXT0+PH374gcmT\nJ9O6dWvs7e05duzYI/Pr3LkzZ8+efeiXp2PGjEGlUmFra8uAAQMICQkpvep/Frq6ugQGBtK2bVt6\n9+5dZsMOExMT2rdvj6+vb2nr3xkzZlBUVISdnR02NjbMmDHjqc7Zr18/7OzsaN26NV26dGHhwoXU\nr1+fQ4cOle7jun37diZMmEBqaiqdOnXC3t4eb29v5s2bV2a8qKgozM3N+f777xk9ejTW1tbP/H68\nCKsPXuTbE1fw6/QaQ9pZKGK5hbmMCRtDTkEOQV2DaFSjkWaS1DDRtldNotKiGL1vNLZ1bVnbfS36\n2n8XB1mWmbbzNFsirzKnrw2DXZuq9dwVoTWqIDyMun83f4hJYeL38fRr04gl77VW3NoqUhUxZv8Y\notOiWdV1FW6N3NR23opCtO19yZzrO/NFhy+IvRnL1CNTKZH/vs8qSRKz/8+Gri3NCNyVwN4zaRrM\nVBAqp/DEdKZsP0WH5nVZ0N9OUdRlWeazY5/xx/U/+Kz9Z1WyqD8NUdjVqGeznkx0msi+y/tYFLVI\ncU9dR1uLlR+0wda8FuO3niT2yi0NZioIlUtCag5+m2J4vZ4RwYMdykxEWHFyBT8l/cQ4+3H0bd5X\nQ1lWHKKwq5mXtRdDWg1h07lNfH32a0XMUE+H9V5O1KtpwPCQKJLSc9V23qr6xaxQeanrd/JqVh7D\nQqJKN7n599Thbee3se70Ot6xeqfMjmevqucu7JIkGUiSFClJUrwkSWckSZqpjsQqs4lOE+lh0YPF\n0Yv59dKviljdGvqEDnNBS5Lw2hhJ+p2C5z6fgYEBmZmZorgLFYYsy2RmZmJg8HzTDLPzCvHaGElB\nkYqQYc7U+9divwNXDvBF5Bd0Mu/E9LbTX5kFSI/z3F+eSvffyeqyLOdKkqQLRAATZFn+o7znVMUv\nT/+tQFXA6H2jiU+P5yvPr3Bp4KKIx13NZuDaP2huVoOto1zLLIN+GkVFRaSkpJSZMy0ImmRgYIC5\nuTm6us+2OC+/SMXgdSc4lZLDN8NdaGtZRxGPT49nxO8jeN3kddZ1X4ehrqE60q7QnvTLU7XOipEk\nyZD7hd1PluUT5R33KhR2gJyCHLx/8ybtbhqhb4aWaRO6/9wNRn4djbuVKf8d6lSmcZEgvKpUJTJj\nN8fy+9k0Vg10oJddA0U8OSeZIb8OoaZeTb75zzfUNnj06tyq4qXOipEkSVuSpDjgJrDvYUVdkqRR\nkiRFS5IUnZ6ero7TVnjG+sYEewZjqGOIX5gfaXeVs2G6vlGPuf1sOXQhnek7T4tbKYLA/ds4s38+\ny29n0vi0V6syRT3jXga+Yb5oSVqs8VzzyhT1p6GWwi7LskqWZXvAHHCRJMnmIceslWXZSZZlJ1NT\nU3WctlKoX70+QZ5B5BXl4Rfmx+3C24r4QJcmjO/SnO+iU1gW9qeGshSEimNteBIhx5IZ0aEZwzs0\nU8TyivIYu38sWflZrOqyisY1G2soy4pNrZ/9ZVnOBg4BPdU5bmXXonYLlnVeRvLtZCYcmEChqlAR\nD+hmxbuO5izf/ydbI69oKEtB0LxdcanM+/U8ve0aMO0/yoVNRSVFfHz4Y85nnWeR+yJsTW01lGXF\np45ZMaaSJNX635+rAZ6A2PzzX9o2aMsctzlE34hmesT0MguYvnjbFg8rU6b/mMCB8zc0mKkgaMax\nixlM/D6ets1q8+V7rdHSUi5Amn18NhGpEcxwnYFHYw8NZlrxqeOKvQFwUJKkU0AU9++x//yY57yS\neln2IsAxgN+Sf2NJ9BJFTFdbi6BBDrzRwIixm08Sf7Vy98kWhKdx7vptRn8TQ7O61Vk71Al9HWWL\n3eD4YHZe3Mlou9G8Y/WOhrKsPJ67sMuyfEqW5TayLNvJsmwjy/IsdSRWVQ2zHsbAlgMJPRvKprOb\nFLHq+jps8HamTg09fEKiSM64q6EsBeHluZZ9j2EbozDU1yZkmAvG1ZTTI3f8uYPg+GD6Nu/LWPux\nGsqychHz614ySZKY7DyZrk26sjBqIb8n/66ImxkZEOrjQoks470xkszc51/AJAgVVc69Irw3RnK3\noJiQYS40rFVNEQ9PCWfW8Vm4NXIjsF2gWID0hERh1wBtLW3md5yPvZk9U49MJTpNOaf/NdMarPNy\n5npOPj6h0eQVvryNjAXhZSkoVjHq62guZdzlqyGOvNFAuWvVmYwzTDw8ESsTK5Z4LEFX69XZhex5\nicKuIQY6BqzsshJzI3PGHxzPxVsXFXHHpiasGNiG0ynZ+H97kmJVSTkjCULlU1Ii8/F38Zy4lMXi\nd1vTvnldRfzqnauM2T+G2ga1CfIMeiVWlaqTKOwa9GABk762Pn77/bhxVzkbpod1fWb+nw37z99k\nxq4zYgGTUGXM+/UcP5+6zpQ3W/J/9srNMLLys/AL80Mlqwj2DKZutbrljCKURxR2DWtUoxHBnsHc\nLrjNmP1juFOo3MJtiGtTxnR6jS2RV1h14GI5owhC5bEh4hL/PXIJr3ZNGe1uqYjdK76H/35/0u6m\nsarLKpoZNytnFOFRRGGvAFrWbsnSzktJyk4i4FAARaoiRXxSjxa83aYRX+5L5PvoqxrKUhCe357T\n15n9y1l6WNcj8C1rxZehxSXFfBL+CaczTrOg4wLszew1mGnlJgp7BdG+YXtmus3kxPUTzDg2o8wC\npvn97ejQvC5Td5zmcOKr0WtHqFoiL2Xx4bY4HJuYsPz9Nmj/awHSvBPzOHT1EFPbTqVr064azLTy\nE4W9AunzWh/GtxnPL0m/sDx2uSKmp6NF8GAHrOoZ4bcphoTUnHJGEYSK588bdxgRGoW5STX+O9QJ\nA13lAqR1p9fxXeJ3+Nj4MLDlQA1lWXWIwl7BjLAdwXtW77EhYQNbzm9RxIwMdNk4zBkTQz28N0Zx\nNStPQ1kKwpO7cTsf741R6OloEzrMBZPqeor47r92s+LkCnpZ9mKCwwQNZVm1iMJewUiSxLS20+jU\nuBPzTsxj/+X9ini9mgaE+jhTpCrBa2Mkt+4WljOSIGjenfwivDdGkZ1XSMgwZxrXVk5bPHbtGJ8d\n/Yy2Ddoyu/1stCRRktRBvIsVkLaWNgvdF2JrasvkI5M5efOkIt7czIh1Xk6k3LrH8NAo8otUGspU\nEMpXWFyC36ZY/rxxh6DBjtg0MlbEz2ed56NDH2FZy5KlnZaiqy0WIKmLKOwVVDWdaqzqsor61esz\nbv84knKSFHFni9osH2DPyavZjN9yElWJmOMuVByyLDNl+ykiLmYw73+dS//pWu41/ML8MNIzIqhr\nEEZ6RhrKtGoShb0CMzEwIdgzGB0tHfz2+ZGep5wN86ZtAwJ7t2Lv2RvM/EksYBIqjkW/X2DHyVQ+\n7mbFu07KzTByCnLwDfOlQFVAcNdg6lWvp6Esqy5R2Cu4xkaNCfIM4lbBLcbuH8vdImXHx2FuzRjl\nbsnXxy+z5nBSOaMIwsvzzR+XCTr0FwNdmjCuS3NFrEBVwPgD40m5k8LyzstpbtK8nFGE5yEKeyVg\nXceaLz2+JPFWIh8d+oiiEuUCpik9W/JW64Ys+O08O0+maChLQYC9Z9L4bFcCnm+YMfv/lAuQVCUq\nph6ZSuzNWL7o+AXO9Z01mGnVJgp7JdHRvCOftfuMY9eO8fmxzxW3XbS0JBa/a0c7yzpM+v4UEX9m\naDBT4VUVc/kW/ltOYmteixUD26Cj/Xd5kWWZRdGL2Hd5H5OcJtHTQuye+SKJwl6J9Hu9H2Psx7D7\nr92sPLlSEdPX0WbNEEdeM62B76YYzlwTC5iElycpPZcRoVHUNzZgvZcThno6injomVA2n9vMkFZD\nGGo9VENZvjpEYa9kfO186f96f/57+r98d+E7Rcy4mi4hPs4YGegwbGMUKbfEAibhxUu/U4DXxki0\nJInQYS7UraGviP966Ve+jPmSHhY9mOg0UUNZvlpEYa9kJEniU9dPcTd3Z+6JuRy8clARb2BcjZBh\nLtwrUpUuDBGEF+VuQTE+IVFk3ClkvbczFnWrK+JRaVFMj5iOYz1H5naYKxYgvSTiXa6EdLR0WOS+\niFa1W/FJ+CfEp8cr4i3qG7F2iBNXMvMY9XWMWMAkvBBFqhLGfhvLmWs5rPqgDfaNayniibcSmXBg\nAk2MmrC883L0tfXLGUlQN1HYKylDXUNWdV2FqaEp4/aPIzknWRFv91odvnyvNZHJWXz0XRwlYgGT\noEayLDN952kOXUhnbj9bur6hnIuedjcNvzA/qulUI9gzGGN943JGEl4EUdgrsTrV6rDGcw0SEr5h\nvmTcU86Geat1Qz7t9QZ7Tqcx+5ezYgGToDbLwv7ku+gUxndpzkCXJorY7cLb+IX5cbfoLkGeQTSo\n0UBDWb66RGGv5JrUbMLqrqvJys9i7P6x5BUpvzAd0dESH7dmbDyazLojlzSUpVCVbIu6wvL9f/Ku\nozkB3awUsUJVIR8e/JDk28ks67yMFrVbaCjLV5so7FWArakti9wXcT7rPB8f/rjMAqZPe71BL9sG\nzN1zjt3x1zSUpVAVHDx/k2k7E3C3MuWLt20VC5BK5BI+jfiUqLQoZrvNxrWBqwYzfbWJwl5FeDT2\nYIbrDCJSI5h9fHaZBUxfvtcal2a1mfhdPMf/ytRgpkJldSolmzGbY3mjgRFBgxzQ1VaWj6UxS/k1\n+Vc+dPiQ3pa9NZSlAGoo7JIkNZYk6aAkSeckSTojSZLolK8h71i9w2i70ey8uJPg+GBFzEBXm/8O\ncaJpHUNGfRPN+bTbGspSqIyuZObhExJFnRp6bPB2poa+cgHS5nObCTkTwvst3sfHxkdDWQoPqOOK\nvRj4WJblNwBXYKwkSa3UMK7wDMbaj6Vv874ExwezPXG7ImZsqEuIjwuGetp4b4jies49DWUpVCaZ\nufcXIBWXyIT6uGBmZKCI77u8jwWRC+japCtTXKYobs8ImvHchV2W5euyLMf+7893gHNAo+cdV3g2\nkiQR2C4Qt0ZuzP5jNuEp4Yp4o1rV2OjtQm5BMd4bosi5V1TOSIIA9wpVDA+N5lr2PdZ7OfGaaQ1F\nPPZGLFPCp2Bnasf8jvPR1tIuZyThZVLrPXZJkiyANsAJdY4rPB1dLV2WeCzBysSKiYcnkpCRoIi3\naliTr4Y4kpSRy+hvoikoFguYhLKKVSX4bzlJfEo2y99vg2PT2op4UnYS/gf8aVijIau6rMJAx6Cc\nkYSXTW2FXZKkGsB24ENZlsvcwJUkaZQkSdGSJEWnp6eXHUBQK0NdQ4I8g6htUJux+8dy5fYVRdyt\neV0WvdOaP5KymPj9KbGASVCQZZnPdp8h7NwNZvaxpqdNfUU8PS8dvzA/dLV0CfYMppZBrXJGEjRB\nLYVdkiRd7hf1zbIs73jYMbIsr5Vl2UmWZSdTU9OHHSKoWd1qdQn2DKZELsEvzI+s/CxFvG+bRkzu\n2ZKf4q8x/7fzGspSqIiCDv3F5hNX8PV4jaHtLBSx3MJcxuwfw62CW6z2XI25kblmkhTKpY5ZMRKw\nHjgny/KS509JUKdmxs1Y2WUlN/JuMG7/uDILmHw9LBnarilrw5PYeFQsYBJgR2wKi36/QF/7hnzS\nQ7nAqEhVxEeHPuLPW3+ypNMSrOtYayhL4VHUccXuBgwBukiSFPe///6jhnEFNbE3s2eB+wLOZJ7h\nk/BPKC4pLo1JksRnb1nTw7oes34+y57T1zWYqaBpR/5M55MfTtH+tTosfKc1Wlp/z3CRZZnPj3/O\n8evH+bz953Ro1EGDmQqPoo5ZMRGyLEuyLNvJsmz/v//2qCM5QX26NunKNJdpHE45zNwTcxULmLS1\nJJa/3waHJiZ8uC2OyEtZjxhJqKrOXMvBb1Mszc1qsGaII3o6yvKw8uRKdv+1u3RKrVBxiZWnr5AB\nLQcwwnYEPyT+wNpTaxUxA11t1g11wtykGiNCo/jzxh0NZSloQsqtPLw3RlHTQIeQYS7UNNBVxL+7\n8B3/Pf1f+r/en9F2ozWUpfCkRGF/xYxvM563LN9iVdwqfrz4oyJmUl2P0GEu6Olo470xihu38zWU\npfAyZecV4rUhkoIiFSE+LtQ3Vk5bPHjlIHNPzMXd3J1PXT8VC5AqAVHYXzGSJDGz/UxcG7gy89hM\njqYeVcQb1zYkZJhz6V/2O/liAVNVll+kYuTX0VzNusfaoU5Y1TNSxOPT4/kk/BNa1W7FIvdF6Gjp\nlDOSUJGIwv4K0tXWZWmnpTQ3aU7AoQDOZp5VxG0aGRM82JGLN3Px3RRDYXGJhjIVXiRViUzAtjii\nkm+xZEBrXC3rKOKXb1/Gf78/poamrOq6CkNdQw1lKjwtUdhfUTX0arC662pq6ddiTNgYUu6kKOLu\nVqbM72/H0YuZTN5+SmzSUcXIsszsn8/ya0Ian/Z6g952DRXxjHsZ+O7zBWCN5xrqVKvzsGGECkoU\n9leYmaEZazzXUFRShF+YH7fybyni7ziaM7G7FTtPprLw9wsaylJ4EdYduUTIsWR83JoxoqOlIpZX\nlMe4/ePIuJfBqq6raFKzSTmjCBWVKOyvOMtalqzsspJrudfwP+DPvWJlx8exnZvzQdsmBB/6i2+O\nJ2skR0G9dsdfY+6ec/SybcCnvd5QxIpLipl4eCLnss6x2GMxdqZ2GspSeB6isAs41HNgvvt8TqWf\nYkr4FFQlfzcFkySJWX2s8XzDjMDdZ/j9TJoGMxWe1/G/Mpn4XTwuzWrz5XtlFyDN/mM2R1KP8Knr\np3g09tBgpsLzEIVdAKBb025MdpnMgasHmBc5T3FPXUdbi5UDHWhtXovxW04Sc1ksYKqMzqfdZtQ3\n0TStY8h/hzhhoKtssbsmfg07/tzBKLtRvGv1roayFNRBFHah1KA3BjHMehjbLmxjfcJ6Rayanjbr\nvZxoYGzA8NBo/krP1VCWwrO4nnMP7w1RGOppE+LjgrGhcgHSzj93EhQfRJ/X+jDOfpyGshTURRR2\nQeFDxw95s9mbLI9dzk9//aSI1amhT6iPCzpaEl4bIrl5Ryxgqgxy7hXhvSGK3IJiNnq70KhWNUX8\nSMoRZh6fSfuG7fm8/ediAVIVIAq7oKAlaTHHbQ4u9V0IPBrI8WvHFfGmdaqz3suZzNxCfELuFwuh\n4iooVjH6m2iSMnL5aogjrRrWVMTPZJzh48MfY2VixZJOS9DV0i1nJKEyEYVdKENPW49lnZfRrFYz\nAg4FcD5L2au9deNaBA1y4Nz1O4zZHEuRSixgqohKSmQmfn+KP5KyWPROa9ya11XEr965ypj9YzDR\nN2F119VU162uoUwFdROFXXgoIz0jgroGUUO3BmPCxnAt95oi3rmlGV/0syE8MZ2pO06LBUwV0ILf\nzvNT/DU+6dmCvm2U2xDfyr+FX5gfKllFcLdgTA3F5jdViSjsQrnqV6/PGs815Bfn4xvmS05BjiI+\nwLkJE7q+zg8xKSzdl6ihLIWHCTl6ia/Ckxji2hQ/j9cUsXvF9xh3YBxpd9NY2WUllsaW5YwiVFai\nsAuP1NykOcu7LCflTgrjD4ynQFWgiH/o+ToDnBqz4sBFvj1xpZxRhJfp19PXmfnzWbq3qsfnfawV\nX4aqSlRMDp/M6fTTLOi4gDZmbTSYqfCiiMIuPJZzfWe+6PAFsTdjmXpkapkFTHP62dCphSmf/nia\nsLM3NJipEJWcxYRtcbRpXIsVA9ug/a8FSPMi53Hw6kGmuEyha9OuGsxUeJFEYReeSM9mPZnoNJF9\nl/exKHqR4p66rrYWqz9wwLqhMeO2xHLyyq1HjCS8KBdv3mFEaDTmtaqx3su5zAKk9Qnr2XZhG8Ns\nhvHBGx9oKEvhZRCFXXhiXtZeDH5jMJvPbSb0TKgiVl1fhw3ezpgZ3V/AdCnjroayfDXdvJ2P14Yo\ndLW1CPVxwaS6niL+018/sTx2OW82e5MPHT7UUJbCyyIKu/BUJjlPoodFD76M+ZI9ScqtbU2N9AkZ\n5owsy3hvjCQjt6CcUQR1upNfhPfGKG7lFbLR25nGtZV9049fO07g0UBc6rswx20OWpL4a1/Vif+H\nhaeiJWkxt8NcHOs5Mv3odCKvRyrilqY1WO/tzI3b+QwPiSKvUCxgepGKVCWM2RzLhRt3CBrkgK25\nsSJ+Pus8AYcCaFarGcs6L0NPW6+ckYSqRBR24anpa+uzvPNyLGpaMOHgBBJvKac6OjQxYeVAB06n\n5jB2cyzFYgHTCyHLMpO3n+LInxnMe9uWTi3MFPFrudcYEzaGGro1COoahJGeUTkjCVWNKOzCMzHW\nNybYMxhDHUP8wvxIu6ts59utVT1m97Xh4IV0Pv0xQSxgegG+3JvIjthUAjyteM+psSKWU5CDX5gf\n+cX5rPFcQ/3q9TWUpaAJorALz6x+9foEeQaRV5SHX5gftwtvK+KD2jZlXOfmbI26yor9FzWUZdW0\n+cRlVh28yECXxozv2lwRK1AVMP7AeK7eucryLstpbtK8nFGEqkoUduG5tKjdgmWdl5F8O5kJByZQ\nqCpUxD/ubkV/B3OWhiXyXdRVDWVZtew7e4MZPybQpaUZs//PRrEAqUQuYeqRqcTejOWLDl/gXN9Z\ng5kKmiIKu/Dc2jZoyxy3OUTfiGZaxDRK5L/vqUuSxPz+tnR8vS5Td57m4IWbGsy08ou9cgv/LbHY\nNjJm1Qdt0NH++6+wLMssilrEvsv7mOg0kZ7NemowU0GT1FLYJUnaIEnSTUmSEtQxnlD59LLsRYBj\nAL8n/86X0V8qYrraWgQPdqRlfSPGbo7lVEq2hrKs3C5l3GVEaDT1ahqw3tsZQz0dRfzrs1+z6dwm\nBr8xmKGthmooS6EiUNcVewggLg9eccOshzGw5UC+Pvs135z9RhGroa/DRm9nTAz18AmJ4kpmnoay\nrJzS7xTgteH+1NLQYS7UraGviP966VcWRy+mW9NuTHKeJDbLeMVJ6pqtIEmSBfCzLMs2jzvWyclJ\njo6OVst5hYpFVaLi48Mfc+DKARZ5LKKHRQ9F/OLNXN5ZcwwTQz22+7WndvUqMq9aluHoMsh+tkZo\nt+PTuHsx86GxElkm5dY9CotLMDepVqZVQHpJAYcLM6itpYeHbl20RVGv0Gr7fox+2x6PP/AhJEmK\nkWXZ6bHHvazCLknSKGAUQJMmTRwvX76slvMKFU9+cT6j9o0iISOBtd3W4lRf+XsYnZzFoHUnaNWw\nJqA6m0UAABg/SURBVN+OcKWannY5I1Uil4/BxjfBoBZoP90uRDl/SVw7pIOWnszDFoXK3L9/rvWQ\ngq2SIEeS0AKMS2RESa/4GgUGUP3t0c/03ApX2P9JXLFXfTkFOQz5dQgZ9zL4uufXZabc/ZaQht/m\nGDzfqMeawY6KLoSV0tZBcPkoBJwFPcPHH/8/d/84wZWRIzFs3ZrG69ehpf/3LRZZlpm64zRbo64y\nt58Ng9o2VTw37W4ag/cMpkQuYfN/NtOgRgO1vRyhYnrSwi5mxQgvxIMFTPra+vjt9+PGXWU73542\n9fn8LWv2nb3BZ7sr+QKmrCQ4/ws4DX+qop5/IZEUf3/0LZpivnqVoqgDrNh/ka1RVxnXuXmZon6n\n8A5j9o8htyiXIM8gUdQFBVHYhRemUY1GBHsGc7vgNmP2j+FO4R1F3Ku9BaM9LNn0xxWCDv2loSzV\n4MRXoKUDLiOf+ClFaWlcHTUKrWrVaPzVV2gbK3u8fBd1laVhifR3MOfj7laKWKGqkA8Pfsil7Ess\n6bSElrVbquVlCFWHuqY7bgGOAy0kSUqRJGm4OsYVKr+WtVuytPNSkrKTCDgYQJGqSBGf3KMlfe0b\nsuj3C2yPSdFQls/hXjbEfgM2/cHoyZbtq27f5urIUZTk5tJ47VfoNmyoiB+8cJOpO0/T8fW6zO9v\nW2YB0qdHPyUyLZJZbrNo37C9Wl+OUDWopbDLsjxQluUGsizryrJsLsvyenWMK1QN7Ru2Z6bbTE6k\nneDTo58qFjD9f3t3Hld1sT5w/DOHg+wCgohsivu+JLlkamWLmi3eylZTTEhyqWyz5V5t8Za5pNcN\nRQU17y1t+Vmmma1WLgjuaGKiAuICKojsB+b3B5adcsE4Cxyf9z91mPl+zzOv9GmY7zzfMRgU797f\nkRua+vHSx7vYkJptx0j/hm1LoawAejxVpe4VpaVkjh5DyeHDhMyehWsr89n2rsxcRi3fRqtAL+Y9\n1gVnJ/O/ojOSZ7D20Fqevu5p7mp6l8WGIRyLLMUIm7i76d2M7TyWNYfWMGPbDLO2OkYDsUO60CzA\nk5j3k9lzNO8Sd6lhyk2VyzCNe0HDjlfsrisqODb+ZQoTEwn69yQ8evQwa08/VcjwhK34utchftj1\neLqYFyAt37ec+JR4Hmz5IE+0k1+KxaVJYhc2M6L9CAa3GEz8nnj+u++/Zm11XZ1JiOyKt5szkQlb\nyThdCwqY9q2Cs5nQY1SVup+cOo2za9ZQ/7lxeN9lPts+XVDK0PhEyso1S4Z3JaCuq1n7+iPrmZw4\nmZtDb+blri9LAZK4LEnswmaUUrzS7RVuCr2JdxLf4esjX5u1B3q7kjC8KyVl5QyLTyS3sPQSd6oB\ntIZNc6BeU2h+5WKT00uXcXrxYnwfeQS/ESPM2opKyxmxZCtHc4tYNDSCZgGeZu3bTmxj/IbxtK/f\nnsm9J+NkcIB9/8KqJLELm3IyOPFu73dpX789438cz/aT283aWzTwYsHjEWScLmLEkiSKy8rtFOkV\nZCTC0WToHgOGy/81OrvuK068/TZet91Kg1dfMZttl1donv5gO9szcvnPQ52IaFzP7Nq03DTGfDuG\nhp4NmX3LbNyMblYZjnAsktiFzbkZ3Zh9y2wCPQIZ/c1o0vLSzNq7N/Fj+oMdSU4/wzMf7KC8ogbu\ncd88p7LKtNMjl+1WmJxM1gsv4NapE0FTpqCcLsy2tdZM/CyFr/aeYMLANvRrZ74XPbswm5ivYzAa\njMy7dR6+rr5WGYpwPJLYhV34uvoy79Z5GA1GYtbHkF1ovhtmYIcgXruzDV+mHOfN1XtrVgHTmcOw\n73OIiIQ6HpfsVnLwIBlPjcI5KIiQuXMwuJqvm8/74SDLNh/hyd5NGNYz3KytoKyAUd+M4kzJGeb2\nnUuol/kJSUJcjiR2YTehXqHMvXUuZ0rOVFZRlp4za3/ixnBG3BhOwsbDLNiQdom72MGWBaAM0DX6\nkl3KTp4kIyoa5exM6MI4jL7ms+1PtmXy7pf7ubtjEC/1M9/yWFZRxrjvx5F6JpVpfabR1r+tVYYh\nHJckdmFXbf3aMq3PNA6cOcC478f9pYDplQGtGdihIW+v/YVVO47aKco/KD5buXe97SCoG3TRLuXn\nzpHx5EhMubmExsZSJyTErP2nAzm8+NEuejTxY8oDHTD84T05WmsmbpzIxqyNTOgxgV4hvaw6HOGY\nJLELu+sV0osJPSaw6dgmJm6aaLbsYjAopg3uSLfwejy/cicbf82xY6TA9mVQmg/dL16QpEtLOTr2\naUpSUwmZOQO3duaz7ZSsPEa+n0yzAE/mP94FF6P5DpdZ22fx2cHPeKrjUwxqPshqwxCOTRK7qBEG\nNR/EU52e4rODnzFr+yyzNhejEwsejyDc34MnlyWz79jZS9zFyspNsCUWwm6A4Ov+0qy15tg//0XB\nxo00fPNNPHuZz7aP5hYRGb8VL1cj8ZHXU9fV/PW+K/avIG53HPc1v4+RHUdadSjCsUliFzXGyA4j\nua/5fcTtjmPF/hVmbd5ulQVMHi5GIuO3kpVbZPsAf1ldeZDGJQqSsmfOJG/VKvzHjsHnH+az7bzC\nMoYuTqSorJyEyK409Dbftvhd+ndM2jKJXsG9eK37a1KAJKpFEruoMZRSvNb9NXqH9GbSlkl8m/6t\nWXuQjxsJw6+noMTEsPhE8orKLnEnK9k8F3wbQ8v+f2k688GHnIqdj88DD+AfE2PWVlxWTtTSJNJP\nFbJgSAQtA73M2red2MaLG16kVb1WTO0zFaPB/FUCQlwtSeyiRjEajEzpPYU29drw0oaX2Jm906y9\nVWBd5j/ehUM5BUQvTaLEZKMCpswkyNhSubb+p8rP/G+/5fgbb+DZpw+BE/5l/jbGCs1zK3aSePg0\nUwd3pEdTP7Nr957ay6hvRhHoEcicvnNwd676+9yFuBRJ7KLGcXd2Z3bf2dR3r8/ob0ZzKO+QWfsN\nTf2Z+kBHthw6zbgVO6mwRQHTpjng4g2dHjX7cdGOHRwd9xyubdsS/N50lNF8tv3WF/v4YvcxXh3Q\nmrs7mu+iSctNY+T6kXjV8SLu9jj83fytPgxxbZDELmokPzc/Ym+NxaAMPLHuCdJyzfex39MpmJf7\nt+KLXcf495p91g0mNwP2roIuQ8HlwntcSg8fJiPmKYwBAYTGzsPgbj7bXvhjGot/PkRkz8aM6GVe\ngHQ47zBR66MwKANxt8cR6FG1d7kLURWS2EWNFVY3jEW3L6JCVxC5LpL9p/ebtUf3bsKwGxqz8KdD\nLPzRigVMifMr/9ntwgHEppwc0qMqC5TC4hZg9DNfYvlsZxZvfbGPAe0D+eedbcyWZw6cOcCwL4dh\nqjCx4PYFNKprfuydENUliV3UaM18m5HQLwGjwcgTXz1BSk7K721KKf45sA392gby1hf7WL0ry/IB\nlORD8lJocw94VxYaVRQUkDEyBlN2NqGx86jTyDwxbzp4iudX7KRr43pMH9zJrAAp5VQKkesicVJO\nxPeLp4Wv+bF3QliCJHZR4zX2bkxCvwQ8jB5Erovkh4wffm9zMihmPNSJiEa+jPtwJ5vTTln2y7cv\nh5I86DEaAG0ykTluHMV79xL83nTcOpofsPHL8bNEL0uikZ87cY9H4Op84UHrhswNDP9yOJ7OniT0\nT6CJdxPLxirEeZLYRa0Q6hXK+wPeJ9w7nLHfjTU7qMPV2YmFQyMIredG9NIkUk/kX+ZOV6GiHLbM\ng9BuENIFrTXHX3+dgh82EDhhAl4332zWPSu3iGGLt+Jex4mE4V3xdr9QgLR833LGfDuGRnUbsbT/\nUnmpl7AqSeyi1qjvXp/4O+LpHdKbtxPfZuLGiRSbigHwca9DQmRXXJydGLo4keN5xdX/wv1rKt/k\neL4gKWfuXHJXfoRfzEh8Hxxs1jWvqIxh8YkUlJhIiOxKsE9lAVJJeQmvb3qddxLfoU9IHxL6JRDg\nHlD92IS4DEnsolZxd3Znxk0ziGofxccHPmbI2iGkn00HILSeOwmR13P2fJI9W1zNAqZNc8EnDFoN\nJPfjj8mZNRvve++l/tixZt3OlZgYnrCVQzkFzB/ShdYN6wJw5OwRHv3iUT5K/Yjh7Ybz3k3vyT51\nYROS2EWt42RwYux1Y5nTdw7HCo4xePVgVqauRGtN2yBvYod04deT5xixJIn8v5vcj26D9I3QLYZz\nP/3MsX9NwKNnTxq++YbZDpdzJSaGLU5kR0Yu/3moMzc080drzScHPmHw54M5XnicOX3n8GyXZ+VI\nO2EzkthFrdU7pDcrBq6gnV873tj0BlHro8jMz6RX8/pMf7ATyUfO8HDcZnLOlVz9zTfPhTpeFNXp\nQuYzz+LSsgXBM2einC+sm586V8KQRVvYnpHL7Ic70799QzLyM4j6KooJGyfQxq8NH931Eb1Deltw\n1EJcmbLHyTQRERE6KSnJ5t8rHJPWmpWpK5mWNA1ThYkhbYYwov0ItqYVErM8mQZ1XYl97MISyRXl\nHYWZHSht8hiH5yVjcHGh0Qf/wzngwtr4gRP5DF+ylez8EmY+1JmezT1ZvGcxy/Yuw8ngxLgu47i/\nxf0YlMydhOUopZK11hFX6id/6kStp5RicMvBrLp3Fbc3vp1FexZx56d3kl6+lkWR7SkqLefeOT/z\nv8T0qh2xl7gAU7EmY0kKmEyELoz7PalrrfnvlnTumv0TRaUVJAzvyEm1ngGfDCBudxy3hN3C/93z\nfwxuOViSurAbi8zYlVL9gJmAE7BQa/3O5frLjF1YU0pOCu8lv8eW41vwcvbizvB/sG1PC5IPOtGj\niR9v3tuWZgFeF7+45BwVU9qQ/kMAxSdNhMUvxv26ynevp57I5/XPU/j511N0bV5OpzaprD78Cfml\n+XQL7MazEc/S1k+OsRPWU9UZe7UTu1LKCUgFbgMyga3Aw1rrvZe6RhK7sIU9OXuI3xPP1+lfU6Er\nCHJtSdbRFhSebcqAFp2JvDGczqE+Zg9D9eb5ZL76Nuey3AmeOQOv225jZ2Yei35MY23qdty90wgM\n2s+x4v0YlIG+YX0Z2nYoHet3vEwkQliGLRN7D2Ci1vqO859fBtBav32paySxC1s6UXCCtYfW8nna\n56SeSQVAmzwpLw7CU4XSvF4jWvg3JNjLh+vffQ6XlAq23d+PNe3D+CXnCIVk4OSahTJWHrbdwrcF\nA5sMpH94f3l5l7CpqiZ2S7zRPxjI+MPnTKCbBe4rhEU08GjAsHbDGNZuGMcLjrMpaxObsray7XgK\nJ4t+YHeJid1H4Z5NFdyYUsGqborlzb+GYlAeRoJdw+jc8CZuCOpKj6AeksxFjWeJxH6xM7z+8muA\nUioaiAYICwuzwNcKcfUCPQIZ1HzQ7wdFmypMZBfmcPh/H+LzfSzFzSB4zFT+Uy+Q1g2C8HfzlxON\nRK1jiT+xmcAfX3wRAvzlNXta6wXAAqhcirHA9wpRbUaDkbq70vCZEYd7QAktXxhL54gB9g5LiGqx\nxH6srUBzpVS4UqoO8BDwmQXuK4TVFe/bR+aYsbjUdyXklhIM3SLtHZIQ1VbtGbvW2qSUGg2so3K7\n42KtdcoVLhPC7sqyssiIfhKDhzuh3X/BqVskuHrbOywhqs0ii4da6zXAGkvcSwhbKM/LIz0qmori\nYhqP7YXzwV3QbaS9wxLCIqQ0TlxzKkpKyBg1irL0dEJmTMUl62NodSfUC7/yxULUAvK4X1xTdEUF\nWS+NpygpmaBpU/GocwCKzvx+QpIQjkBm7OKacnLyZPK//JKAF1/Eu39/2DwPgjpDWHd7hyaExUhi\nF9eMU/EJnF6yFN/Hh1Avchj8uh5OHaicrauLlWMIUTtJYhfXhLNr1nBy8mS87riDBuPHV74fZtMc\nqBsMbe6xd3hCWJQkduHwChITyXppPG4RXQh6dzLKYIDje+DQD9A1Gpycr3wTIWoRSezCoZUcOEDm\n6DE4h4UROns2BheXyobNc8HZHboMtW+AQliBJHbhsMpOnCA9KhqDiwthC+bj5ONT2ZB/AnavhE6P\ngpuvfYMUwgpku6NwSOX5+WRERVORn0+j95fhHBx8oXHrQigvg+4x9gtQCCuSxC4cji4tJXPMWErS\n0gidH4tr69YXGsuKIGkRtOwPfk3tF6QQViSJXTgUXVFB1iuvUrh5M0GT38GzZ0/zDrs+hMJT0GOU\nfQIUwgZkjV04lOzp0zm7ejX1n30W73v+tI1R68qCpMAO0KjnxW8ghAOQxC4cxun3l3Nq4SJ8Hn4I\nv+iov3Y4+A1k/yIFScLhSWIXDuHs+vWcmDQJz759CXztNbMDqn+3aQ54BkLbQbYPUAgbksQuar3C\nbdvJev4F3Dp0IHjqFJST0187ndgLB7+FbtFgrGP7IIWwIUnsolYrSTtEZkwMzoGBhMTOw+DmdvGO\nm+eC0Q26yAlJwvFJYhe1lik7m4yoKDAaCV0Yh9H3EsVG57Jh1wro9DC417NtkELYgWx3FLVS+bkC\n0p98EtOZMzRasoQ6oaGX7py0CMpLoPtTtgtQCDuSxC5qHV1WxtFnnqFkfyqh8+bi1r7dpTuXFVdW\nmja/A/yb2y5IIexIEruoVbTWHPvXBAp++omGk97Cs3fvy1+weyUUZEtBkrimyBq7qFVyZs0i79NP\n8R89Gp/77rt8598Kkhq0g/Ar/A9ACAciiV3UGmc+XEHO3Hn4PHA//qOqsF6e9j2cTKmcrUtBkriG\nSGIXtUL+d99x/PXX8ejTm8AJEy5egPRnm+aARwC0u8LMXggHI4ld1HhFu3ZxdNxzuLZuTcj06Shj\nFR4NZe+vPNO0axQYXawfpBA1iCR2UaOVHjlCxpMjMfr7Ezo/FoOHR9Uu3DwXjK4QMdy6AQpRA1Ur\nsSulHlBKpSilKpRSEZYKSggA06lTpEdFg9aELpiP0d+/ahcWnIKdH0CHB8GjitcI4UCqO2PfA/wD\n2GCBWIT4XUVhIRkjYzCdPElo7DxcwsOrfnHSYjAVS0GSuGZVax+71nofULUHWUJUkTaZODruOYpT\nUgiZPQu3Tp2qfrGpBLbGQbNbIaCV9YIUogarVQVKOS8+St5PO+wdhrAybdKUndUE3uSK176XYd9V\nXGwqhnMnoEes1eIToqa7YmJXSn0NBF6k6VWt9aqqfpFSKhqIBggLC6tygH/kFNAAl4bef+taUbv4\nNamL7/UBf+/i1ndBk5stG5AQtYjSWlf/Jkp9DzyvtU6qSv+IiAidlFSlrkIIIc5TSiVrra+4UUW2\nOwohhIOp7nbHQUqpTKAH8IVSap1lwhJCCPF3VXdXzKfApxaKRQghhAXIUowQQjgYSexCCOFgJLEL\nIYSDkcQuhBAORhK7EEI4GIsUKF31lyqVDRyx+RdXnz+QY+8gbEzG7PiutfFC7R1zI611/St1skti\nr62UUklVqfpyJDJmx3etjRccf8yyFCOEEA5GErsQQjgYSexXZ4G9A7ADGbPju9bGCw4+ZlljF0II\nByMzdiGEcDCS2P8GpdTzSimtlHL4k5KVUlOUUr8opXYppT5VSvnYOyZrUUr1U0rtV0r9qpQab+94\nrE0pFaqU+k4pte/8ofRP2zsmW1FKOSmltiulVts7FmuQxH6VlFKhwG1Aur1jsZH1QDutdQcgFXjZ\nzvFYhVLKCZgD9AfaAA8rpdrYNyqrMwHPaa1bA92BUdfAmH/zNFd36GKtIon96r0HvAhcEw8ntNZf\naa1N5z9uBkLsGY8VdQV+1Vqnaa1LgQ+Ae+wck1VprY9prbed//d8KhNdsH2jsj6lVAhwJ7DQ3rFY\niyT2q6CUuhs4qrXeae9Y7GQ4sNbeQVhJMJDxh8+ZXANJ7jdKqcZAZ2CLfSOxiRlUTs4q7B2ItVTr\noA1HdLnDu4FXgNttG5H1VeXAcqXUq1T+6r7clrHZkLrIz66J38qUUp7Ax8AzWuuz9o7HmpRSA4GT\nWutkpdRN9o7HWiSx/4nW+taL/Vwp1R4IB3YqpaBySWKbUqqr1vq4DUO0uEuN+TdKqaHAQKCvdtz9\nsZlA6B8+hwBZdorFZpRSzlQm9eVa60/sHY8N9ATuVkoNAFyBukqp97XWj9k5LouSfex/k1LqMBCh\nta6NLxKqMqVUP2A60EdrnW3veKxFKWWk8uFwX+AosBV4RGudYtfArEhVzlCWAKe11s/YOx5bOz9j\nf15rPdDesViarLGLK5kNeAHrlVI7lFKx9g7IGs4/IB4NrKPyIeIKR07q5/UEhgC3nP9vu+P8TFbU\ncjJjF0IIByMzdiGEcDCS2IUQwsFIYhdCCAcjiV0IIRyMJHYhhHAwktiFEMLBSGIXQggHI4ldCCEc\nzP8DcvoV+siEdUYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x110d95940>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = gluon.loss.Huber(rho=0.5)\n",
    "\n",
    "with ag.record():    # start recording\n",
    "    theloss = loss(output, thelabel)\n",
    "theloss.backward()   # and compute the gradient\n",
    "\n",
    "plt.plot(output.asnumpy(), theloss.asnumpy(), label='Huber loss 0.5')\n",
    "plt.plot(output.asnumpy(), output.grad.asnumpy(), label='Gradient of Huber loss 0.5')\n",
    "\n",
    "# and now for the same loss function with rho=1.0, the default\n",
    "loss = gluon.loss.Huber()\n",
    "\n",
    "with ag.record():    # start recording\n",
    "    theloss = loss(output, thelabel)\n",
    "theloss.backward()   # and compute the gradient\n",
    "\n",
    "plt.plot(output.asnumpy(), theloss.asnumpy(), label='Huber loss 1')\n",
    "plt.plot(output.asnumpy(), output.grad.asnumpy(), label='Gradient of Huber loss 1')\n",
    "\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Quantile Regression\n",
    "\n",
    "In most cases we want to find an output $y$ which is in some way maximal for a given $x$, e.g. the one with the smallest amount of variance, the most likely one, etc. But there are cases where this isn't quite the most desirable thing: imagine that we want to build a tool for physicians to assess whether a child is of normal height. *normal* is obviously relative -  relative to age, gender, the ethnic background of the parents, etc. While a good physician might have a good intuition, it would be great if we could quantify this. That is exactly what *quantile regression* does. It aims to estimate some output $f(x)$ such that $\\Pr(y \\leq f(x)|x) = \\tau$ for some quantile $\\tau$. This allows us to trace quantile curves for all sorts of probabilities, such as the table below computed by the CDC.\n",
    "\n",
    "![](img/growth-2-20-girls.png)\n",
    "\n",
    "To calculate such a table we can use a skewed loss function. Statisticians often call it a 'pinball loss', since it looks like the levers on a pinball machine. Basically it's an L1 loss that has been tilted to one side or another. \n",
    "\n",
    "$$l(y,f) = \\begin{cases}\n",
    "\\tau (y-f) & \\text{ for } f<y \\\\\n",
    "(1-\\tau) (f-y) & \\text{ otherwise}\n",
    "\\end{cases}$$\n",
    "\n",
    "Depending on how far we tilt this loss, we end up with a loss function that underweights (small $\\tau$) or overweights (large $\\tau$) errors on the left or on the right. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd8zdf/wPHXJ0skJCSxg4hEyF6IhFhFi6o9ixg1aitV\n/Ep1Kr6SBrX3VpVWS7U1QykZYsQKEsRMQkJkJ+f3R7g1QiJucjPO8/HweMjN53M+73td73vu+Zzz\nPooQAkmSJKnk0NJ0AJIkSZJ6ycQuSZJUwsjELkmSVMLIxC5JklTCyMQuSZJUwsjELkmSVMLIxC5J\nklTCyMQuSZJUwsjELkmSVMLoaOKiZmZmwsLCQhOXliRJKrZCQkJihRCVcjtOI4ndwsKC4OBgTVxa\nkiSp2FIU5VpejpNDMZIkSSWMTOySJEkljEzskiRJJYxGxtil/ElPTyc6OpqUlBRNhyJJ+aavr4+5\nuTm6urqaDqXEkom9GImOjqZ8+fJYWFigKIqmw5GkNyaEIC4ujujoaOrUqaPpcEostQ3FKIqirSjK\nSUVRfldXm9LzUlJSMDU1lUldKrYURcHU1FR+6yxg6hxjHwecV2N7Ug5kUpeKO/keLnhqSeyKopgD\nHYAV6mjvVcJuxLPk0JWCvIQkSVKxp64eux/wKZD1qgMURRmmKEqwoijBMTEx+brIjtBoZv9xgdX/\nROYzTOltRUdH88EHH2BtbY2lpSWjR48mNTVV7dc5ePAgR48eVf28ZMkS1q1bB4CPjw/bt2/Pc1tf\nfPEF8+bNU3uMAHv27MHGxgYrKytmz56d4zHz58/H1tYWR0dHWrduzbVreVpjIkn59taJXVGUjsA9\nIUTI644TQiwTQrgLIdwrVcp1RWyOZnS0pa1tFWb9do6Ak9H5akPKPyEEXbt2pXPnzkRERBAREUFy\ncjKffvqp2q/1YmIfMWIEAwYMUPt13kZmZiajRo3ijz/+4Ny5c2zevJlz5869dJyLiwvBwcGcPn2a\n7t27F8jrJUnPUkeP3QvopChKFLAFaKUoygY1tPsSHW0t/Pu44FnXlEk/nWbf+bsFcRnpFfbv34++\nvj6DBg0CQFtbG19fX9atW0diYiJr1qxh9OjRquM7duzIwYMHARg5ciTu7u7Y2dkxc+ZM1TEWFhbM\nnDkTV1dXHBwcuHDhAlFRUSxZsgRfX1+cnZ05fPjwK3vdISEhNG/eHDc3N9q1a8ft27df+xzCwsLw\n8PDA0dGRLl268ODBAwD8/f1VverevXsDcOjQIZydnXF2dsbFxYVHjx4919aJEyewsrLC0tISPT09\nevfuza+//vrSNVu2bImBgQEAHh4eREfLTolUsN56uqMQYiowFUBRlBbAJCHEh2/b7qvo62qzbIA7\nfZf/y8cbQ1k3uBGNLU0L6nJF1qzfwjl366Fa27StbsTM9+1e+fvw8HDc3Nyee8zIyAgLCwsuX778\n2ra/+eYbTExMyMzMpHXr1pw+fRpHR0cAzMzMCA0N5ccff2TevHmsWLGCESNGUK5cOSZNmgTAvn37\nXmozPT2dMWPG8Ouvv1KpUiW2bt3K9OnTWbVq1SvjGDBgAAsWLKB58+bMmDGDWbNm4efnx+zZs4mM\njKRMmTLEx8cDMG/ePBYtWoSXlxeJiYno6+s/19bNmzepWbOm6mdzc3OOHz/+2tdh5cqVvPfee689\nRpLeVrFceVqujA6rfRpSo2JZhq4NJvxWgqZDKhWEEDnOaBBC5Hrutm3bcHV1xcXFhfDw8OeGLLp2\n7QqAm5sbUVFReY7n4sWLnD17ljZt2uDs7MzXX3/92t5wQkIC8fHxNG/eHICBAwcSGBgIgKOjI/36\n9WPDhg3o6GT3d7y8vJg4cSL+/v7Ex8erHn/d837djI8NGzYQHBzM5MmT8/wcJSk/1LpASQhxEDio\nzjZfxbRcGTYMaUz3xUcZuOoEP43wpI6ZYWFcukh4Xc+6oNjZ2fHzzz8/99jDhw+5e/cuNjY2nD17\nlqys/+6fP52rHBkZybx58wgKCqJixYr4+Pg8N4+5TJkyQPbQTkZGRp7jEUJgZ2fHsWPH3uZpAbBr\n1y4CAwPZuXMnX331FeHh4Xz22Wd06NCB3bt34+Hhwd69e6lfv77qHHNzc27cuKH6OTo6murVq+fY\n/t69e/nmm284dOiQ6vlKUkEplj32p6pXKMv6oY3JEvDhiuPcSZCLHgpS69atSUpKUs1OyczM5JNP\nPmH06NGULVsWCwsLwsLCyMrK4saNG5w4cQLITv6GhoYYGxtz9+5d/vjjj1yvVb58+ZfGtF9kY2ND\nTEyMKrGnp6cTHh7+yuONjY2pWLEihw8fBmD9+vU0b95cFW/Lli2ZM2cO8fHxJCYmcuXKFRwcHJgy\nZQru7u5cuHDhufYaNmxIREQEkZGRpKWlsWXLFjp16vTSdU+ePMnw4cPZuXMnlStXzvW5S9LbKtaJ\nHaBupXKsHdSI+KQ0+q88zoPHaZoOqcRSFIWAgAC2b9+OtbU1pqamaGlpMX36dCB76KJOnTo4ODgw\nadIkXF1dAXBycsLFxQU7OzsGDx6Ml5dXrtd6//33CQgIUN08zYmenh7bt29nypQpODk54ezs/NxM\nmpysXbuWyZMn4+joSFhYGDNmzCAzM5MPP/wQBwcHXFxcmDBhAhUqVMDPzw97e3ucnJwoW7bsS2Pj\nOjo6LFy4kHbt2tGgQQN69uyJnV32N6kZM2awc+dOACZPnkxiYiI9evTA2dk5x+QvSeqk5GV8VN3c\n3d2FujfaOHolFp/VQdhWM2Lj0MYYlil5ZXDOnz9PgwYNNB2GytGjR+nTpw87dux46aaqJL1OUXsv\nFxeKooQIIdxzO67Y99if8qxrxoI+LpyOjmfEhhBSMzI1HVKJ5+npybVr12RSl6QipsQkdoB2dlX5\nvpsjhyNimbj1FJlZhf9tRJIkSdNK3HhFD/eaJCSn8/Wu8xiV1eHbLg6y6JAkSaVKiUvsAEObWXL/\ncRo/HrxCRQM9Pn23fu4nSZIklRAlMrEDTG5nQ3xyuiq5f+RtqemQJEmSCkWJTeyKovDVB/YkJKfz\nze7zGBvo0tO9Zu4nSpIkFXMl6ubpi7S1FHx7OtPM2ozPfj7NnrN3NB1SsXf37l369u2LpaUlbm5u\nNGnShICAgLdq89kCXzNmzGDv3r35aicsLIzdu3e/8Xl9+vTB0dERX1/fl363bNky6tevT/369XF3\nd1cVNVO3b7/99rmfPT09AYiKisLe3v6N2ipXrpza4npWamoqvXr1wsrKisaNG+dY/uHpQq8GDRpg\nZ2fHDz/8UCCxSLkQQhT6Hzc3N1GYElPSRedFR4T1tN3in8sxhXptdTp37pxGr5+VlSU8PDzE4sWL\nVY9FRUUJf3//l45NT0/Pc7szZ84Uc+fOfev4Vq9eLUaNGvVG59y+fVvUqlUrx9/99ttvwtXVVcTE\nZL9nQkJCRI0aNUR0dPRbx/oiQ0PDHB+PjIwUdnZ2amnrbS1atEgMHz5cCCHE5s2bRc+ePV865tat\nWyIkJEQIIcTDhw+FtbW1CA8Pf+k4Tb+XiysgWOQhx5boHvtThk+KhlmYGfDR2mBOR8drOqRiaf/+\n/ejp6TFixAjVY7Vr12bMmDEArFmzhh49evD+++/Ttm1bEhMTad26taok77Mlbb/55htsbGx45513\nuHjxourxZzfReFVJ3hYtWjBlyhQaNWpEvXr1OHz4MGlpacyYMYOtW7fi7OzM1q1bn4s9JSWFQYMG\nqVaXHjhwAIC2bdty7969HFe4fv/998ydOxczMzMAXF1dGTRoEIsWLQKySw7HxsYCEBwcTIsWLYDs\ncr6enp64uLjg6empen5r1qyha9euvPvuu1hbW6vqsn/22WckJyfj7OxMv379gJx73ZmZmUyePJmG\nDRvi6OjI0qVLX/vvJYRg8uTJ2Nvb4+DgoHpNbt++jbe3N87Oztjb23P48GEyMzPx8fFRHZvTt5df\nf/2VgQMHAtC9e3f27dv3UiG0atWqqVYcly9fngYNGnDz5s3XximpX4kdY39RBQM91g1uTPclR/FZ\nHcS24U2wqlwwX1kLxR+fwZ0z6m2zqgO8l/MuQJBdtvfpf9pXOXbsGKdPn8bExISMjAwCAgIwMjIi\nNjYWDw8POnXqRGhoKFu2bOHkyZNkZGTg6ur60iKn3EryZmRkcOLECXbv3s2sWbPYu3cvX375JcHB\nwSxcuPCluJ4m4zNnznDhwgXatm3LpUuX2LlzJx07diQsLCzH5/tiXO7u7qxevfq1r0H9+vUJDAxE\nR0eHvXv3Mm3aNFXxtLCwME6ePEmZMmWwsbFhzJgxzJ49m4ULF+YYw7NWrlyJsbExQUFBpKam4uXl\nRdu2balTp06Ox+/YsYOwsDBOnTpFbGwsDRs2xNvbm02bNtGuXTumT59OZmYmSUlJhIWFcfPmTc6e\nPQugKl38rGfLFOvo6GBsbExcXJzqg+9FUVFRnDx5ksaNG7/2eUnqV2oSO0BVY33WD2lMjyVH6b/y\nONtHelKjQllNh1VsjRo1iiNHjqCnp0dQUBAAbdq0wcTEBMjuMU6bNo3AwEC0tLS4efMmd+/e5fDh\nw3Tp0kW1+UROtVOeLckL2b3VatWqqX7/pqV+jxw5ovpmUb9+fWrXrs2lS5cwMjJ6o+f8Yg81JwkJ\nCQwcOJCIiAgURSE9PV31u9atW2NsbAyAra0t165de66m++v89ddfnD59WvWNJiEhgYiIiFcm9iNH\njtCnTx+0tbWpUqUKzZs3JygoiIYNGzJ48GDS09Pp3Lkzzs7OWFpacvXqVcaMGUOHDh1o27Ztnp77\nq9aIJCYm0q1bN/z8/N74NZbeXqlK7AB1zAxZO7gRvZf+S/+Vx/lpeBNMyxXDMqqv6VkXlBfL9i5a\ntIjY2Fjc3f8rXWFo+F/p5I0bNxITE0NISAi6urpYWFioyvXmtmhM5FKS901L/eYlIb/I1taWkJAQ\nWrVqpXosNDRU9Xx1dHRUZYqfLUP8+eef07JlSwICAoiKilIN0Twb95vE/uxzWLBgAe3atcvz8Tnx\n9vYmMDCQXbt20b9/fyZPnsyAAQM4deoUf/75J4sWLWLbtm0vbVjytEyxubk5GRkZJCQkqD7En5We\nnk63bt3o16+f6gNYKlylYoz9RXbVjVnp05CbD5LxWR3Eo5T03E+SaNWqFSkpKSxevFj1WFJS0iuP\nT0hIoHLlyujq6nLgwAHVJs7e3t4EBASQnJzMo0eP+O233146901L8sLrS/16e3uzceNGAC5dusT1\n69exsbF5bXuffvopU6ZMIS4uDsgeRgkICGD48OFA9hh7SEj2Vr/PfuAlJCRQo0YNIHtcPS90dXWf\n69nnpF27dixevFh13KVLl3j8+PErj/f29mbr1q1kZmYSExNDYGAgjRo14tq1a1SuXJmPPvqIIUOG\nEBoaSmxsLFlZWXTr1o2vvvqK0NDQl9rr1KkTa9euBWD79u20atXqpQ9oIQRDhgyhQYMGTJw4MU/P\nXVK/UpnYARrVMWHxh66cu/2QYetCSEmXRcNyoygKv/zyC4cOHaJOnTo0atSIgQMH8v333+d4fL9+\n/QgODsbd3Z2NGzeqNqlwdXWlV69eODs7061bN5o1a/bSufkpyduyZUvOnTuX483Tjz/+mMzMTBwc\nHOjVqxdr1qzJdcOLTp06MWTIELy8vLCysqJp06b88ssvPN2MfebMmYwbN45mzZqhra2tOu/TTz9l\n6tSpeHl5kZmZt/fVsGHDVLs4vcrQoUOxtbXF1dUVe3t7hg8f/toef5cuXXB0dMTJyYlWrVoxZ84c\nqlatysGDB1X7uP7888+MGzeOmzdv0qJFC5ydnfHx8eG77757qb0hQ4YQFxeHlZUV8+fPZ/bs7G+N\nt27don379gD8888/rF+/nv3796v2i83PFFTp7ZSYsr359cvJm4zfGkYb2yos7ueKjnbR/ayTpU41\nJyMjg0GDBpGVlcWGDRtk/aG3JN/L+ZPXsr2lboz9RZ1dahCflMYXv53jsx1nmNvdUf6nlV6io6PD\n+vXrNR2GJOVJqU/sAD5edXiQlM4P+yKoaKDLtPYNZHKXJKnYkon9ifHvWBOflMbyw5FUMNBjVEsr\nTYckSZKULzKxP6EoCjPftyM+OZ25f16kooEefRvX0nRYkiRJb0wm9mdoaSnM6+HEw+R0pv9yBuOy\nunRwrJb7iZIkSUVI0Z0CoiG62lr82M8N99oVGb/1JIGXYjQdkiRJ0huRiT0HZfW0WTGwIXUrlWP4\n+hBCrz/QdEhFRnR0NB988AHW1tZYWloyevRoUlNT1X6dgwcPPjdvfcmSJaxbtw54vlBYXjxbFljd\n9uzZg42NDVZWVqp53TnZtm0btra22NnZ0bdv3wKJRZKekon9FYzL6rJuSCMqG5Vh0OogLt3NeUVj\naSKEoGvXrnTu3JmIiAgiIiJITk5WVSlUpxcT+4gRIxgwYIDar/M2MjMzGTVqFH/88Qfnzp1j8+bN\nnDt37qXjIiIi+O677/jnn38IDw/Hz89PA9FKpYlM7K9Rubw+G4Y0poyOFv1XHufG/Vcvny8N9u/f\nj76+PoMGDQKya534+vqybt06EhMTWbNmDaNHj1Yd37FjR9XGFCNHjsTd3R07OztmzpypOsbCwoKZ\nM2eqSvteuHCBqKgolixZgq+vr6qc7qt63a8q7fsqYWFheHh44OjoSJcuXXjwIPvbmL+/P7a2tjg6\nOtK7d28ADh06pFo96eLi8lK5ghMnTmBlZYWlpSV6enr07t37udLETy1fvpxRo0ZRsWJFACpXrpzb\nSy1Jb0XePM1FTRMD1g9pTM+lx7KLho3wpFJ5zRcN+/7E91y4f0GtbdY3qc+URlNe+fucytgaGRlh\nYWHB5cuXX9v2N998g4mJCZmZmbRu3ZrTp0/j6OgIgJmZGaGhofz444/MmzePFStWMGLECMqVK8ek\nSZMA2Ldv30tt5lbaNycDBgxgwYIFNG/enBkzZjBr1iz8/PyYPXs2kZGRlClTRlWydt68eSxatAgv\nLy8SExPR19d/rq1ny9hCdpGs48ePv3TNS5cuAahKDHzxxRe8++67r329JOltyB57HthULc8qn4bc\nfZjKgFUnSEgunUXDhBA5LtzKS1mKbdu24erqiouLC+Hh4c8NWbxpCd6nni3t6+zszNdff010dPQr\nj09ISCA+Pp7mzZsDMHDgQAIDAwFUdVo2bNiAjk52f8fLy4uJEyfi7+9PfHy86vHXPe+cXp+MjAwi\nIiI4ePAgmzdvZujQoTnWO5ckdZE99jxyq12RJf3dGLo2iKFrg1g3uDFl9bRzP7GAvK5nXVBeLNsL\n8PDhQ+7evYuNjQ1nz55VlbGF/0rZRkZGMm/ePIKCgqhYsSI+Pj7Plbl90xK8T+VW2vdN7Nq1i8DA\nQHbu3MlXX31FeHg4n332GR06dGD37t14eHiwd+9eVSEz+K+M7VPR0dFUr179pbbNzc3x8PBAV1eX\nOnXqYGNjQ0REBA0bNnzruCUpJ7LH/gaa16vE/J7OBF97wKhNoaRnZuV+UgnSunVrkpKSVLNTMjMz\n+eSTTxg9ejRly5bFwsKCsLAwsrKyuHHjBidOnACyk7+hoSHGxsbcvXuXP/74I9drva4E71NvWtrX\n2NiYihUrqrbAW79+Pc2bN1fF27JlS+bMmUN8fDyJiYlcuXIFBwcHpkyZgru7OxcuPD/01bBhQyIi\nIoiMjCQtLY0tW7bkuGlI586dVVvxxcbGcunSJSwtLXN9DSQpv2Rif0PvO1Xn68727L9wj8k/nSIr\nq/CrY2qKoigEBASwfft2rK2tMTU1RUtLi+nTpwPZQxd16tTBwcGBSZMmqbbRc3JywsXFBTs7OwYP\nHoyXl1eu13r//fcJCAjIcS/Sp/JT2nft2rVMnjwZR0dHwsLCmDFjBpmZmXz44Yeq/VAnTJhAhQoV\n8PPzw97eHicnJ8qWLct77733XFs6OjosXLiQdu3a0aBBA3r27ImdnR0AM2bMYOfOnUB2HXVTU1Ns\nbW1p2bIlc+fOxdTUNNfXQJLy663L9iqKog8EAmXIHtrZLoSY+bpzilLZ3vxadOAyc/+8iI+nBTPf\nty2UomFFrdTp0aNH6dOnDzt27HjppqokvU5Rey8XF4VZtjcVaCWESFQURRc4oijKH0KIf9XQdpH1\ncYu63H+cxsojkVQ00GPcO9aaDqnQeXp6qnZFkiSp6HjrxC6yu/yJT37UffKnxI9PKIrC9PYNiE9K\nx3fvJSoY6DLQ00LTYUmSVIRFJkRSxzjnzcfVSS1j7IqiaCuKEgbcA/4WQrw0mVdRlGGKogQrihIc\nE1My6q9oaSl8382BNrZVmLkznF/Dbmo6JEmSiiAhBKvOrqLzr53Zd/3lNRnqppbELoTIFEI4A+ZA\nI0VR7HM4ZpkQwl0I4f50z8iSQEdbiwV9XGhcx4RPtp3iwIV7mg5JkqQiJDkjmSmHp+Ab4kub2m1o\nUq1JgV9TrbNihBDxwEGgVC2r09fVZsVAd+pXK8+IDSEERd3XdEiSJBUBtxNvM/CPgeyJ3MM413HM\n9Z6Lga5BgV/3rRO7oiiVFEWp8OTvZYF3APWudS8GyuvrsmZQI2pUKMvgNUGcu/VQ0yFJkqRBIXdD\n6L2rNzce3WBh64UMdRhaaFtuqqPHXg04oCjKaSCI7DH239XQbrFjVq4M64c2plwZHQasOkFU7GNN\nh6R2d+/epW/fvlhaWuLm5kaTJk0ICAh4qzafLfA1Y8YM9u7dm692wsLC2L179xuf16dPHxwdHfH1\n9X3pd8uWLaN+/frUr18fd3d3VVEzdfv222+f+9nT0xOAqKgo7O1fGtl8rXLlyqktrmelpqbSq1cv\nrKysaNy48SvLP8THx9O9e3fq169PgwYN1LIyuLjZdnEbQ/8cipGeERs7bMTb3LtwAxBCFPofNzc3\nUZJF3H0onGf9Kbxm7xN3EpLV1u65c+fU1lZ+ZGVlCQ8PD7F48WLVY1FRUcLf3/+lY9PT0/Pc7syZ\nM8XcuXPfOr7Vq1eLUaNGvdE5t2/fFrVq1crxd7/99ptwdXUVMTExQgghQkJCRI0aNUR0dPRbx/oi\nQ0PDHB+PjIwUdnZ2amnrbS1atEgMHz5cCCHE5s2bRc+ePXM8bsCAAWL58uVCCCFSU1PFgwcPXjpG\n0+/lgpKWkSZmHZ0l7NfYi5F/jxQJqQlqbR8IFnnIsTKxF5Cw6w+E7ed/iLbzD4kHj1PV0qam/zPs\n3btXeHt7v/L3q1evFt27dxcdO3YULVu2FI8ePRKtWrUSLi4uwt7eXvzyyy+qY7/++mtRr1490bp1\na9G7d29VYh84cKD46aefhBBCBAcHC29vb+Hq6iratm0rbt26JYQQonnz5uLTTz8VDRs2FNbW1iIw\nMFCkpqaKmjVrCjMzM+Hk5CS2bNnyXGzJycnCx8dH2NvbC2dnZ7F//34hhBAODg5CX19fODk5icDA\nwOfOadq0qdi3b99zj/3f//2fmDp1qhBCiNq1a6uSflBQkGjevLkQQojjx4+LJk2aCGdnZ9GkSRNx\n4cIF1evTpUsX0a5dO2FlZSUmT54shBBiypQpQktLSzg5OYm+ffsKIf5Lzs8m9oyMDDFp0iTh7u4u\nHBwcxJIlS3L8d3h6blZWlpg0aZKws7MT9vb2qtfk1q1bolmzZsLJyUnY2dmJwMBAkZGRIQYOHKg6\ndv78+S+127ZtW3H06FEhRPYHt6mpqcjKynrumISEBGFhYfHS4y/S9Hu5IMQkxYgBuwcI+zX2wi/E\nT2RkZqj9GnlN7LIIWAFxqlmBZQPcGbQ6iMFrgtgwtDEGeup7ue98+y2p59V7K6NMg/pUnTbtlb8P\nDw9XlQl4lWPHjnH69GlMTEzIyMggICAAIyMjYmNj8fDwoFOnToSGhrJlyxZOnjxJRkYGrq6uL61c\nza0kb0ZGBidOnGD37t3MmjWLvXv38uWXXxIcHMzChQtfimvRokUAnDlzhgsXLtC2bVsuXbrEzp07\n6dixI2FhYTk+3xfjcnd3Z/Xq1a99DerXr09gYCA6Ojrs3buXadOmqYqnhYWFcfLkScqUKYONjQ1j\nxoxh9uzZLFy4MMcYnrVy5UqMjY0JCgoiNTUVLy8v2rZtS506Oc+L3rFjB2FhYZw6dYrY2FgaNmyI\nt7c3mzZtol27dkyfPp3MzEySkpIICwvj5s2bnD17FiDH6pPPlinW0dHB2NiYuLg4zMzMVMdcvXqV\nSpUqMWjQIE6dOoWbmxs//PADhoaGr31uxV14bDjjDowjITWBOd5zeK/Oe7mfVIBkrZgC5GVlhn8f\nZ8JuxDN8fQhpGSWraNioUaNwcnJ6rkphmzZtMDExAbK/DU6bNg1HR0feeecdbt68yd27dzl8+DBd\nunTBwMAAIyOjHAtn5VaS901L/R45coT+/fsD2Ym3du3aqjrpb0LkoQRHQkICPXr0wN7engkTJjxX\nmKx169YYGxujr6+Pra3tG63c/euvv1i3bh3Ozs40btyYuLg4IiIiXnn8kSNH6NOnD9ra2lSpUoXm\nzZsTFBREw4YNWb16NV988QVnzpyhfPnyWFpacvXqVcaMGcOePXswMjLK03N/8WZgRkYGoaGhjBw5\nkpMnT2JoaPjaLQNLgt+v/s7APQPRUrRY9946jSd1kGV7C9y79tX4rqsDU34+w8RtYfzQ2wVtrbe/\nM/66nnVBebFs76JFi4iNjcXd/b/SFc/2zDZu3EhMTAwhISHo6upiYWGhKteb2+wAkUtJ3jct9ZuX\nhPwiW1tbQkJCaNWqleqx0NBQ1fPV0dFRlSl+tgzx559/TsuWLQkICCAqKooWLVq8FPebxP7sc1iw\nYAHt2rXL8/E58fb2JjAwkF27dtG/f38mT57MgAEDOHXqFH/++SeLFi1i27ZtL21Y8rRMsbm5ORkZ\nGSQkJKg+xJ89xtzcnMaNGwPQvXv3EpvYM7My8Qv1Y034GtyquDG/xXxM9E1yP7EQyB57IejVsBZT\n36vP76dvM+PXs/lKMkVBq1atSElJYfHixarHkpJevV1gQkIClStXRldXlwMHDqh6p97e3gQEBJCc\nnMyjR4+6HotmAAAgAElEQVT47bffXjr3TUvywutL/Xp7e7Nx40Yge0ej69evY2Nj89r2Pv30U6ZM\nmUJcXByQPYwSEBDA8OHDgext/UJCQgCe+8BLSEigRo0aAKxZs+a113hKV1eX9PTXb+DSrl07Fi9e\nrDru0qVLPH786plX3t7ebN26lczMTGJiYggMDKRRo0Zcu3aNypUr89FHHzFkyBBCQ0OJjY0lKyuL\nbt268dVXXxEaGvpSe506dWLt2rUAbN++nVatWr30AV21alVq1qzJxYsXgeydr2xtbfP0GhQnCakJ\njNo3ijXha+hl04vlbZcXmaQOssdeaIY3r8uDpHSWHLpCRQM9JrV7fVIpihRF4ZdffmHChAnMmTOH\nSpUqYWhoyPfff5/j8f369eP999/H3d0dZ2dn1SYVrq6u9OrVC2dnZ2rXrk2zZs1eOvdpSd6xY8eS\nkJBARkYG48ePV5XFzUnLli2ZPXs2zs7OTJ06lV69eql+9/HHHzNixAgcHBzQ0dFhzZo1z/Wec9Kp\nUydu3bqFl5cXGRkZ3Llzh1OnTvF05fTMmTMZMmQI3377raqHCtkfCAMHDmT+/PnP9fZfZ9iwYTg6\nOuLq6qr6AHrR0KFDiYqKwtXVFSEElSpV4pdffnllm126dOHYsWM4OTmhKApz5syhatWqrF27lrlz\n56Krq0u5cuVYt24dN2/eZNCgQapvIN99991L7Q0ZMoT+/ftjZWWFiYkJW7ZsAeDWrVsMHTpUNdV0\nwYIF9OvXj7S0NCwtLXO9J1HcXIm/wtj9Y7n1+BYzm8yke73umg7pJW9dtjc/SkLZ3vwQQjB1xxm2\nBN3g/zo0YGizN9tsQZY61ZyMjAxV4tuwYUOhLTQpqYrre/nA9QNMPTIVfW19fFv64lLZpVCvX5hl\ne6U8UhSFb7o4kJCczte7zlPBQI/ubuaaDkvKAx0dHdavX6/pMCQNEUKw7PQyFoYtxM7UDr+WflQ1\nrKrpsF5JjrEXMm0tBb/ezjS1MmPKz6f5+9xdTYckSdJrJKUn8cmhT1gYtpCOlh1Z8+6aIp3UQSZ2\njSijo83S/m7Y1zBm1KZQjl2Jy/O5xfXGqyQ9VZzew9GPovnwjw/Zd30fk9wn8W3Tb9HX0dd0WLmS\niV1DDMvosManIbVMDPhoXTBnohNyPUdfX5+4uLhi9R9Dkp4lhCAuLg59/aKfHI/fPk7vXb258/gO\ni1svZqDdwGJzb0XePNWw2wnJdF98jOT0TH4a0YS6lV5dwCk9PZ3o6Ojn5kxLUnGjr6+Pubk5urq6\nmg4lR0IINl3YxNyguVgYWeDfyp9aRrU0HRaQ95unMrEXAVdjEumx5Bj6utr8NKIJ1SuU1XRIklQq\npWam8tWxr/j1yq+0rNmS75p9h6Fu0SmHkNfELodiigDLSuVYO7gRD5PT6b/yOPcfp2k6JEkqde4l\n3WPwnsH8euVXRjiNwK+lX5FK6m9CJvYiwr6GMSsGuhP9IBmf1SdITM37UnNJkt7O6ZjT9P69NxHx\nEfi28GWU8yi0lOKbHotv5CVQY0tTFvV1JfzWQ4atCyY1I1PTIUlSiRcQEYDPHh/0tPXY0H4D79R+\nR9MhvTWZ2IuYd2yrMLe7I0evxDFucxgZmSWrIqQkFRUZWRl8f+J7ZhydgWsVV7Z02EK9ivU0HZZa\nyMReBHV1NWdGR1v2hN9hekDxLRomSUVVfEo8I/4ewYbzG/iwwYcseWcJFfQraDostZElBYqowU3r\nEJ+Uhv/+y1Qw0GVq++JXV0OSiqKL9y8y7sA4YpJi+Nrraz6w+kDTIamdTOxF2IQ29XiQlM7SwKtU\nNNRjRPO6mg5Jkoq1v6/9zfQj0ymvW541767BoZKDpkMqEDKxF2GKojCrkx3xyenM/uMCFcrq0rtR\n0VgoIUnFSZbI4sewH1l6eimOlRzxa+FHJYNKmg6rwMjEXsRpaSn8r4cTD5PTmRZwBuOyurznUE3T\nYUlSsZGYlsjUI1M5eOMgXay68H8e/4eetp6mwypQ8uZpMaCno8WSD91wqVWRcVvCOBIRq+mQJKlY\nuPbwGv129+Nw9GGmNprKLM9ZJT6pg0zsxUZZPW1WDWyIZSVDhq0PJuzGy7vIS5L0n39u/kOfXX24\nn3KfZW2W0bdB32JTxOttycRejBgb6LJucCPMypXBZ/UJIu7mvL+nJJVmQgjWnF3Dx/s+ppphNTZ3\n2Eyjao00HVahkom9mKlspM+GIY3R1dai/8oTRD949WbSklTapGSkMPXIVP4X8j9a12rN+vfWY16+\n9O1SJhN7MVTL1IB1gxuRlJZB/5UniE1M1XRIkqRxdx7fYeCegey+upsxLmP4X/P/YaBroOmwNEIm\n9mKqQTUjVvk05HZCMgNXneBhSrqmQ5IkjQm9G0qv33tx7eE1/Fv5M8xxWKkZT8+JTOzFmLuFCUs+\ndOPinUcMXRtMSrosGiaVPj9d+okhfw2hvF55NrXfRIuaLTQdksbJxF7MtbCpzPxezgRF3Wf0plDS\nZdEwqZRIz0znq2Nf8eWxL2lcrTGbOmzCsoKlpsMqEmRiLwE6OVXnyw/s2Xv+HlO2nyYrSxYNk0q2\nuOQ4hv41lG2XtjHIfhCLWi3CSM9I02EVGXLlaQnR36M2Dx6nMf/vS1Qw0OPzjg1K9RijVHKdizvH\nuAPjeJDygO+bfU97y/aaDqnIeevErihKTWAdUBXIApYJIX5423alNzemlRUPktJY9U8kFQ10GdPa\nWtMhSZJa7b66m5lHZ1JBvwJr31uLnamdpkMqktTRY88APhFChCqKUh4IURTlbyHEOTW0Lb0BRVH4\nvIMtCUnp/O/vS1Qw1KO/R21NhyVJby0zKxP/k/6sOrsK18quzG8xH9OyppoOq8h668QuhLgN3H7y\n90eKopwHagBqT+wxSTFkikyqGlZVd9MlhpaWwvfdHXmYks6MX89iXFaXTk7VNR2WJOXbw7SHTAmc\nwpGbR+hZryefNfoMXW1dTYdVpKn15qmiKBaAC3Bcne0+9UPoD3QM6IhfiB8P0x4WxCVKBF1tLRb2\ndaWhhQkTt4Zx8OI9TYckSflyNf4qfXf15d9b//K5x+d83uRzmdTzQG2JXVGUcsDPwHghxEtZV1GU\nYYqiBCuKEhwTE5Ova4xwGkGb2m1YdXYV7Xe0Z234WlIz5arLnOjrarNioDs2VcszYkMIIdfuazok\nSXojh24cou/uvjxKe8SKdivoadNT0yEVG4o69tNUFEUX+B34UwgxP7fj3d3dRXBwcL6vd+H+BfxC\n/Pjn1j9UM6zGGJcxtK/THm0t7Xy3WVLFJqbSY8kx4hJT2Tq8CQ2qySlhUtEmhGDFmRUsOLmA+ib1\n+aHlD1QrJ/cgAFAUJUQI4Z7rcW+b2JXsOXVrgftCiPF5OedtE/tT/97+l/nB8zl//zz1KtZjgtsE\nvKp7yWl+L7hxP4keS46RKQQ/j/CklmnprJ8hFX1J6Ul8/s/n/HXtL9rXac8Xnl9QVqespsMqMgoz\nsTcFDgNnyJ7uCDBNCLH7VeeoK7FD9pZXf0b9iX+oP9GJ0TSq2ogJbhOwN7NXS/slRcTdR/RYegwj\nfV22j2hCZSN9TYckSc+5mXiTsfvHEvEgggluE/Cx85GdtBcUWmLPD3Um9qfSM9PZdmkbS08t5UHq\nA9pZtGOsy1hqGck9Qp8KuxFP3+X/UsvEgK3DmmBsIG9CSUVD0J0gJh6cSGZWJnOaz6FpjaaaDqlI\nymtiLzElBXS1denXoB+7u+5mhNMIAqMD+eCXD/j636+JTZZbyQE416zAsv7uXIlJZPDaIJLSMjQd\nklTKCSHYdH4TH/31ESb6JmzuuFkmdTUoMT32F8Umx7Lk1BK2X9pOGe0y+Nj5MMBuAIa6hgV63eJg\n95nbjN4USjPrSiwf4I6eTon5fJeKkbTMNL45/g07InbQwrwF3zX7jnJ65TQdVpFW6oZiXiUqIQr/\nk/78fe1vTPRNGOk0km71uqGrVbqHITafuM7UHWfo5FQdv17OaGnJsUyp8MQkxTDh4AROxZximOMw\nRjmPQkuRHYzcyMT+gtMxp/EN8SX4bjC1ytdijOsY2tVuV6pvziw+eIXv91ygv0dtvvzArlS/FlLh\nORNzhvEHxvMo/RFfe31NW4u2mg6p2Ch1Y+y5cazkyKp2q1jUehF62npMPjSZvrv6cuL2CU2HpjEj\nW9RluLcl6/+9hu/flzQdjlQK7LyyE589Puhq67L+vfUyqReQUlW2V1EUvM298aruxe9Xf2dh2EKG\n/DUErxpeTHCdgI2JjaZDLHSfvVefB0lp+O+/TAUDPQY3raPpkKQSKCMrg/8F/48N5zfQqGoj5jWf\nR0X9ipoOq8QqNUMxOUnNTGXz+c0sP7OcR2mP6GjZkdEuo6lernQVzcrIzGLUplD+DL/L/J5OdHUt\nfbu6SwUnPiWeSYGTOH77OP0a9OMT909K/T2u/JJj7G8gITWBlWdXsvHcRgSCPvX78JHDR1TQr6Dp\n0ApNSnomg9cEcTzyPks/dOMd2yqaDkkqASIeRDB2/1juJt3lc4/P6WLdRdMhFWsysefDncd3+DHs\nR3698iuGOoYMdhhMvwb9Ss2S5sTUDPou/5eLdx6xbnAjGlvKetdS/u29tpdpR6ZRTrccvi19cark\npOmQij2Z2N9CxIMI/EP9ORh9kMoGlRnlPIpOdTuho1Xyb0ncf5xGjyVHufcwlc3DPLCvYazpkKRi\nJktkseTUEhafWoyDmQN+Lf2obFBZ02GVCDKxq0HwnWB8Q305HXOausZ1Gec6jhY1W5T4aYG34pPp\nvvgoaZlZ/DTCkzpmclGXlDeP0x8z7fA09t/YT6e6nZjRZAZltMtoOqwSQyZ2NRFCsO/6Pn4I/YGo\nh1G4VHZhottEnCs7azq0AnUlJpGeS46hr6vN9pFNqGZcOoajpPy7/vA64w6MIzIhkknuk+jXoF+J\n7wQVNpnY1SwjK4OAywH8GPYjscmxtKrZinGu47CsYKnp0ArMmegE+iz/l2rG+mwb3oSKhnqaDkkq\noo7ePMqkwEloKVrMaz4Pj2oemg6pRJILlNRMR0uHHvV6sKvLLsa4jOH4neN02dmFL45+wb2kkrn1\nnIO5McsHuHPtfhKD1gTxOFUWDZOeJ4RgbfhaRu4bSRWDKmzusFkm9SJA9tjz6X7KfZafXs6Wi1vQ\nUXTob9ufQfaDKK9XXtOhqd1f4XcYuTGUJpamrPRxp4yO3KlKgpSMFGYdm8XvV3+nTe02fO31NQa6\nchOXgiSHYgpJ9KNoFpxcwO7I3RiXMWaYwzB61++NnnbJGrbYHhLNpJ9O0d6hKgv6uKIti4aVance\n32H8gfGEx4UzynkUwxyHySJehUAm9kJ2Pu48fqF+HL11lOqG1RntMpoOlh1K1Jt9xeGrfL3rPH0a\n1eTbLg7yxlgpFXYvjPEHxpOckcx3zb6jVa1Wmg6p1JBj7IWsgWkDlrZZyrI2yzAuY8y0I9Po+VtP\n/rn5D5r48CwIQ5tZMqplXTafuMGcPy9qOhxJA36+9DOD/hyEoa4hG9tvlEm9iCr5K24KWZPqTWhc\nrTF7Ivfgf9KfEXtH0LhqYya4T8DO1E7T4b21SW1teJCUzuKDV6hooMsw77qaDkkqBOlZ6cw5MYct\nF7fgWd2TOd5zMC4jF68VVXIopgC9uA/ruxbvMtZlLDWNamo6tLeSmSUYu+Uku07fZk43R3o2LN7P\nR3q9+yn3+eTgJwTfDcbHzodxruNKxSrsokiOsRchiWmJrAlfw7pz60jPTKeHTQ+GOw7HtGzxrcWS\nlpHFkLVB/HM5lh/7ufGufVVNhyQVgAv3LzB2/1jup9znC88v6GjZUdMhlWpyjL0IKadXjtEuo9nV\nZRddrbuy7eI22u9oz+JTi0lKT9J0ePmip6PF0v5uONWswNjNJzl6WW4YXtLsidxD/939yRJZrH13\nrUzqxYjssWtAZEIkC04u4O9rf2Oqb8pIp5F0rde1WNaojk9Ko9fSf4l+kMSmjzxwqll6Sh2XVJlZ\nmSwMW8iKMytwqezC/BbzMStrpumwJORQTLFwKuYUviG+hNwNobZRbca6jKVN7TbFbhrh3YcpdFt8\nlMepGfw0oglWlUveIq3S4lHaI6YETuHwzcN0s+7G9MbT0dUufh2Okkom9mJCCMHhm4fxDfHlcvxl\nHMwcmOA2gYZVG2o6tDcSFfuY7kuOoautsH2kJzUqyKJhxU1kQiRj948l+lE0nzX6jJ42PYtdJ6Ok\nk2PsxcTTfVi3v7+dr7y+4l7SPQb/OZiP937MxfvFZ664hZkh6wY3IjE1g/4rjxOXmKrpkKQ3EBgd\nSN9dfXmY9pDlbZfTq34vmdSLMZnYiwhtLW06W3Xm9y6/M9FtImExYfT4rQfTj0znduJtTYeXJ7bV\njVjl05Bb8ckMXH2CRynpmg5JyoUQghVnVjB632jMy5uzucNm3Kvm2iGUijg5FFNEJaQmsPLMSjae\n3whA3wZ9GeowtFgsCjlw4R4frQvG3aIiawY1Ql9XFg0ripIzkpnxzwz2RO3hXYt3+dLry1KzDWRx\nJcfYS4g7j++w8ORCdl7ZSTndcgxxGEK/Bv3Q19HXdGiv9cvJm4zfGkYb2yos7ueKjrb8cliU3Eq8\nxbgD47h4/yLjXMcx2H6wHHopBmRiL2EuPbiEf6g/h6IPUdmgMqOdR9Opbie0tYpub3jt0Shm7gyn\nu5s5c7o5oiUrQhYJQXeC+OTgJ2RkZTDbezbe5t6aDknKI3nztISpV7EeC1svZFW7VVQ1qMqMozPo\ntrMbB28cLLJFxgZ6WjD+HWu2h0Tz7e7zRTbO0kIIwZYLWxj21zCMyxizscNGmdRLKJnYi5mGVRuy\nof0G5reYT6bIZMz+Mfjs8SHsXpimQ8vRuNbW+HhasOJIJD8evKLpcEqttMw0Zh2bxTfHv8Gzhieb\nOmyijnEdTYclFRC1JHZFUVYpinJPUZSz6mhPej1FUWhTuw07PtjB5x6fc+3hNfr/0Z/xB8YTmRCp\n6fCeoygKMzra0tm5OnP/vMjG49c0HVKpE5scy5A/h/BzxM985PAR/i39S+ROX9J/1DLGriiKN5AI\nrBNC2Od2vBxjV6+k9CTWn1vP6vDVpGSk0MW6CyOdRlLZoLKmQ1NJz8xi+PoQDly8x4I+LnR0rK7p\nkEqF8Nhwxh4Yy6O0R3zp9SXvWryr6ZCkt1CoY+xCiEDgvjrakt6cga4Bw52Gs7vrbnrX780vl3+h\nw44O+If68yjtkabDA0BXW4tFfV1xr12RCVvDOHQpRtMhlXi/XfmNAX8MQEfRYd1762RSLwKEEIVy\nr0mOsZcgJvomfNboM3Z+sJOWtVqy/Mxy2u9oz4ZzG0jLTNN0eJTV02bFwIZYVS7PiPUhhF5/oOmQ\nSqSMrAzmBc1j2pFpOFZyZHPHzdQ3qa/psEqte49SCDgZzcRtYTT+dh+noxMK/Jpqm+6oKIoF8Pur\nhmIURRkGDAOoVauW27Vrcqy1oJ2LO4dviC//3v6XGuVqMMZlDO/VeU/j+7Dee5RCjyXHiE9KZ9vw\nJthUleO96pKQmsDkQ5M5dvsYfer3YXLDycWyamhxlpyWyYmo+xyJiOFwRCwX7mR/a65ooIuXlRkj\nW9TFrnr+FhoW+jz23BL7s+QYe+E6eusofiF+nL9/nvom9ZngOgHPGp4ajenG/SS6LzmKEPDzSE9q\nmhhoNJ6S4PKDy4w9MJbbj2/zf43/j271umk6pFIhK0tw7vZDAiNiOBIRS3DUA9Iys9DT1sLdoiLN\nrCvRzNoM22pGb72WQyZ26TlZIos/Iv9gwckF3Ey8iUc1D8a7jdfoPqwX7zyi59JjVDDQ5acRTahc\nvmivpi3K9l/fz9TDUymrUxa/ln44V3bWdEgl2s34ZFWP/OiVOO4/zh7qrF+1PM2szWhqXYlGFiaU\n1VPvAsJCTeyKomwGWgBmwF1gphBi5auOl4ldc9Iy09h2cRtLTy8lPjWe9+q8xxiXMdQsr5l9S0Ov\nP6Df8uNYmBmyZZgHxmXlsMGbyBJZLD29lB/DfsTe1B7flr5UNZTbFKrbo5R0/r36ZHjlcixXYx4D\nULl8GZpam9HM2gwvK7MC75zIkgLSaz1Ke8Tqs6tZf249GSKDXja9GOY4DBN9k0KPJfBSDEPWBuFc\nswLrBjdWey+npEpKT2L6kensvb6X9y3fZ6bnTMpol9F0WCVCRmYWp6ITOBIRy+GIGMJuxJORJSir\nq01jSxOaWpnhXa8S1pXLFWqNHZnYpTy5l3SPJaeWsCNiB/o6+vjY+TDAdgAGuoU75v376VuM2XyS\nljaVWdrfDV1ZNOy1bjy6wdj9Y7macJVP3D6hv21/WcTrLQghuBaXxOEnwyvHrsbxKCUDRQGHGsbZ\nwytWlXCtXYEyOprreMjELr2RqwlX8Q/1Z9/1fZiVNWOk00i6WHcp1BkVm45fZ1rAGT5wro5vT2dZ\nNOwVjt06xuTAyQghmNt8Lp7VNXsjvLiKT0rjn8txHLmcncyjHyQDUKNCWbzrZSdyz7qmVDTU03Ck\n/5GJXcqXsHth+Ib4EnovFAsjC8a6juWdWu8UWm9w0YHLzP3zIgOb1OaLTnayF/oMIQQbzm9gXvA8\nLI0t8W/pT00jzdwbKY7SMrIIufaAI5ezZ6+cvpmAEFC+jA5N6pqqbnpamBoU2fedTOxSvgkhOBR9\nCL8QP64kXMHRzJEJbhMKZWcdIQTf7DrPiiORjH/HmvHv1CvwaxYHqZmpfHnsS3Ze2Umrmq34ttm3\nGOoaajqsIk0IQcS9RA5HxHIkIoZ/r94nOT0TbS0Fl5oVntz0rISTuXGx2S8gr4ldpzCCkYoXRVFo\nUbMFzWo0Y+eVnSwKW8SgPwfhbe7NeNfxWFe0LtBrT+/QgPjkdPz2RlChrC4+XqW7CuHdx3eZcHAC\nZ2LP8LHTxwx3Gq7xRWZFVcyjVP65HEtgRAz/XI7l7sPsvXctzQzp4W5OM+tKeFiaUF6/ZM++kj12\nKVcpGSlsurCJFadXkJieSKe6nRjlPIpq5aoV2DUzMrP4eGMof527i18vZzq71CiwaxVlYffCmHBw\nAknpSXzb7Fta12qt6ZCKlJT0TE5E3lfd9Hx2laenlRneT4ZXalQoGVv+yaEYSe0SUhNYcWYFm85v\nAqBfg34McRhSYPuwpqRn4rP6BMFRD1g2wI1W9asUyHWKqoCIAL769yuqGFTBv5V/gX5TKi6ervI8\nHBHLkcsxBEU9IC3jv1WeTa3NaGZVCbvqb7/KsyiSiV0qMLcTb7MwbCG/XfmNcnrl+MjhI/rU71Mg\n+7A+Skmn7/LjXLr7iA1DG9PQovDn2Re29Kx05gbNZfOFzXhU82Be83nFYhPzgnIrPjl7PvnlWP65\nHPvcKs+mVmY0tTajcR3TUrH+QSZ2qcBdenAJvxA/Dt88TBWDKoxyHlUg+7DGJabSY+kxYh6lsnVY\nE2yrG6m1/aLkQcoDPjn0CUF3guhv25+JbhPR0Spdt8ISUzP490ocR56MlT9d5VmpfBmaPUnkTa3M\nqGxU+kpQyMQuFZqgO0H4hvhyJvYMVhWsGO86Hm9zb7VOGbsZn0z3xUdJzxRsH9EEC7OSNyPk4v2L\njDswjpikGGZ6zqRT3U6aDqlQZGRmcfpmAocvZQ+vnLyevcpTX1eLxnWypyE2s65EvSqFu8qzKJKJ\nXSpUQgj+vvY3/if9ufbwGm5V3JjgNgGnSk5qu8ble4/oseQYhmV0+HmkJ1XU3GMTaWk82LyZzMeP\n1dpuXlx+cJl91/dRRrsM7S3bF6ndrwpCQlI6UXGPuRaXxI37SaRmZKEoUMVIn1omBtQ2NaB6hbJo\nl8BxcuMOHdCrXTtf58rELmlEelY6Oy7tYPGpxcSlxNGmdhvGuIxR28bJp6Pj6bPsX2pULMu24U2o\nYKC+VYFJwcFc+7C/2tqTpJzUXL6Mcs2a5etcmdgljUpKT2LduXWsPrua1MxUulp3ZaTTSCoZVHrr\nto9ejsVndRB2NYzYOLQxBnrqGYN+/O+/XPcZRK01azBoWPCLsR6lPWL6kekcij5EV+suTG00DT3t\norN8/W2kZWRx8voDjlyO5cjlOM7ejCdLQDk9HTzqmtLU2pSmVkV7lWeB0dLK93OWC5QkjTLQNWCE\n0wh61OvBstPL2HZxG79f/Z3+tv0ZZDeIcnrl8t22p5UZ/n1c+HhjCMPXh7ByYEP0dNSwYCcrCwBF\nVwdFu2BnWEQlRDH2wFiuP7zO1CbT6W3Tu1gnOCEEl5+s8jwcEcPxyPskpWWv8nSuWYHR79jQzNoM\nJ/MKxWaVZ3EmE7tUoEzLmjK18VQ+bPAhC04uYNnpZfx08SeGOw2nZ72e6GrnbwXgu/ZVmd3VkU9/\nPs2EbWH493Z56/FYkfXk22sBr+o8cvMInx76FB0tHZa3XU7Dqg0L9HoF5ekqz6dzyp+u8qxjZkh3\nN3OaWpnhUdcUoxK+yrMokoldKhQ1jWoyp/kcBtoNxDfEl9knZrP+3HrGuozl3Trv5muJfM+GNYlP\nTuPb3RcwLqvLN53t367XK5702Avohp0QgtXhq/EL8aNexXr80OoHapQrPitqn67yPPIkmZ+//RCA\nCk/28nw6FdG8otzmUNNkYpcKlZ2ZHcvbLufYrWP4hvoy5fAU1oSvYYLbBJpUb/LG7Q3zrsuDpHQW\nH7xCRQNdJrern//gngzFoKX+HntyRjIzj87kj8g/aGfRji89vyz0mvdv6ukqzyOXYzkSEcuJqPuq\nVZ5utSsyuV328IpddeMSOXulOJOJXSp0iqLgWcMTj+oe7I7czcKTCxn29zCaVGvCBLcJNDBt8Ebt\nfdrOhvikNBYduEJFAz2GNrPMV1yqiQRqHoq5nXibcQfGceH+Bca5jmOI/ZAiO55+OyH5yTh5LEcv\nxxL3ZJWnTZXy9PeoTTNrMxrVMVHbDWupYMh/HUljtBQtOlp2pG3ttmy9uJVlp5fR8/eetK/TnjEu\nYwOPn/AAAA1oSURBVDAvb56ndhRF4evODiQkp/P1rvNUMNCju1vezn3O0zF2NfY+Q+6GMPHgRNIy\n01jYeiHe5t5qa1sdnl3leTgihivPrPL0rlfpyc5BpXOVZ3EmE7ukcXraevS37U9nq86qfVj/uvYX\nvW1685HjR3nah1VbS8G3lzOPUoKZ8vNpjPR1aGv3hps6q8bY1dNj33ZxG98d/w7z8ub80OoHLI3z\n901CnZ6u8jwSkT28Enr9wXOrPPs0qkVTazNsqpQvst8qpNzJeexSkXMv6R4/hv1IwOUAyuqUZbD9\nYD5s8GGexqQfp2bQb8Vxzt1+yNpBjWhS1zTP133499/cHDOWOr8EoF8//2P16ZnpfHfiO3669BNN\nazTle+/vMdLTXH2ba3GPn2w2Ecs/V2JVe3naVzd+Ug3RDNfaFdHXLflFtIo7uUBJKvauxl/lh9Af\n2H9jv2of1q7WXXMtivXgcRo9lx7jdkIKmz/ywME8b5URH/75FzfHjaPOr7+ib5O/nZtik2P55OAn\nhN4LZYj9EMa4jFF7UbTcJCSlc/RKLIFPpiHeuP/fXp5PqyF6WZlhUoT28pTyRiZ2qcQIuxfG/JD5\nnLx3EgsjC8a5jqN1rdavHSq4k5BCt8VHSU7P5KcRTahbKfcFUQ/37OHm+AlY/raTMtZvXvs8PC6c\ncfvHkZCawCzPWbS3bP/GbeRHWkYWodcfqErbnol+ssqzjA4elqZPNmY2o46ZoRxeKeZkYpdKFCEE\nB28cxC/Uj6sJV3Gs5MhEt4m4VXF75TmRsY/pseQoetpabB/pSfVcdtF5uHs3Nyd+guWu3ylTt+4b\nxbfr6i5mHp2Jib4JP7T84Y1n9ryJZ1d5Hrkcy79X41SrPJ3MjWlmnX3T06lmBXTlKs8SRZYUkEoU\nRVFoWaslzcyf7MN6chE+e3xoYd6Cca7jsKpo9dI5dcwMWTOoEX2W/Uv/lcf5aYTna4cf8rPyNDMr\nE79QP9aEr8G1sivzW8zHtGzex/XzKjbxmVWeEbHceZgCZD/Hbq7mNLU2o4lc5Sk9IRO7VKzoaOnQ\n1bor79V5j43nN7LqzCq6/dZNtQ9rVcPnZ8LY1zBmxUB3Bqw6gc/qE2z6yINyZV7xtn/DlacJqQlM\nCZzCP//f3r3HVlnfcRx/f3toEURAShVbKEIlKHResKCkMAniYFpxyBYFHbqpbIsE0ZEpQuKWzDiH\nDkSRSYyGRZgaGTo3HF4jwhxiEVDu4ADLbRYoiAW59Ls/zjnIVaA95zw8Tz+vhMAJPc/zfcKvH379\nnd9l41xu6ngT93e9v9ZbJBxpz74DzF+7jTmr4mPlh63yLPr2sIk2LU7vRU4SDA3FSKhV7amKn8O6\nfBpZlsXgiwZzR/HR57C+vXQLv3ihnCvateC527secwbIjtdeY+P9D1D0xkxyCtt8533XVK1h+Pv3\nsvHrTYzuOoofd7ixTs9R486yzTvjM1dWb2X+uvgqz+yY0aXw7INB3jm/GTGNk4ebZdV6dbPG2KVe\n2bhrIxMXTuT1Na9zVs5Z8XNYLxpEw1jDg18z45MK7n1pEX07n8vEwV2O2mWw6oXJbPr9OIrKtpDT\n5MBx7/Ve40aMysuloTvjt3zJZd/sTdtzSQTdMh069KnVWzXGLvVKfpN8Hu7xMEM6DWH8gvE8Xv44\nU5dPZdilwyhrX0YsK8aAy1pTVb2P372+lAdnfMqjAy8+fJZI9bb4751/BO2P/vDT3Zm841OeqlpI\np5xcnjinF62KTv6Ivm8OHKBi227Wbf2adduq2bor/h/CmQ0b0Da3MW0TJwc1aahx8kjLTf9CNQW7\nRErHFh2Z1GcSH236iHHl4xgzdwxTlk5hRJcR9Czoyc9K27G9eh8T3lnF2Y1zGHXtIQGe3I+9eAB0\nvf6w61bvq2bM3DG8VbWQsvZlPNT9Ic5o8N3L7A/UOIsrquLTEI9Y5dmtXS49S+Nj5Re20ipPSS0F\nu0RSt/O6Me26aby57k0mLJjA3e/cTcm5Jdx3+X3c26eYquq9PDP7c5o3zuFXveJTG70mMfxyxIKi\niq8qGP7ecNZUrWFkyUiGdBpy3CBev7Wa2au+ZM6qSv69ppKde/YDUFzQlDt7tuf7HbTKU9JPwS6R\nZWb0Pb8vvQt7M33ldCYtmsTgmYPj57BeNZyq6nwe/ddymjfOZlC3wkO27f02dOdtmsfI90dywA/w\n9NVPU1pQetg9kqs8P0hsbbt+WzUA+c3OoF9xK3p2yNMqT8m4lAS7mfUDngBiwLPu/odUXFckFbKz\nsrn5wpvpX9SfKUum8PyS53l3/bvc2GEgpXuuZPSMT2nWKJvuyaGYWAx3Z9ryaYydP5a2TdvyZO8n\nKWxaeNhZnh+sqmRxYpXnmTkxuhflckePdvTo0JL2WuUpAarzrBgziwErgWuACmA+MMjdlx7vPZoV\nI0Gq3F3JM4ue4ZWVr5Ady6bx7t5sWHsFrzaYS9aL/6Tty0/wyFcf8OrqV+nVuhd3XTia8rW7mbOq\nkg8TqzyzDC5t05weiVWel2qVp2RAxqY7mll34Lfu3jfxehSAuz9yvPco2OV0sH7neiZ8MoFZa2eR\nVdOEQbObccOH6/jjg0V87OsoajCAzet7snlHfPbK+bmNE/PJ8+helEuzRpq9IpmVyemOBcAXh7yu\nAK5IwXVF0qqwaSGPXfUYt3e+nUfnPc62Mz8CYPmuCnZvv4X/7u9C6QUt6HFBvFeuVZ4SFqkI9mMN\nJB71Y4CZDQWGAhQWFqbgtiKpUdyymL9c+xxzPrwNmM+tze+ix/W3UFygszwlnFIxKFgBHLr+ujWw\n8cgvcvfJ7l7i7iV5eXkpuK1I6pgZnZqcA8Avu5VwSZvmCnUJrVQE+3ygg5m1M7Mc4Gbg7ym4rkhm\nJT9vimkWsIRbnVuwu+83s2HALOLTHZ9z9yV1rkwk05LTHTN84pFIqqWka+LuM4GZqbiWSFAOrjyN\nKdgl3DTxViQpedDGCc5UFTndKdhFkg6uPNW3hYSbWrBIgh9jrxiRMFKwiyS5gl2iQcEuknSwx65v\nCwk3tWCRhPhQjGu6o4Segl0kyT2+QYbp20LCTS1YJKmmRsEukaAWLJLkNfEd7RTsEnJqwSIJXuNg\nrmCX0AvVErs9K1awb8OGoMuQiNq7ebuGYiQSQhXsVS+9xPZpfw26DImwBo0cdFaphFyogj33zjtp\nNnBg0GVIVC1+mexFEzj22TEi4RGqYM/Ozyc7Pz/oMiSqdrSEFTUaipHQUwsWSUoetKFgl5BTCxZJ\nSu4Vo2CXkFMLFklSsEtEqAWLJCnYJSLUgkWSNMYuEaEWLJKkHrtEhFqwSNLBYNc8dgk3BbtIkoJd\nIkLBLpLkWpwk0aBWLJKkYJeIUCsWOUhb9ko0qBWLJKnHLhGhViySpGCXiAjV7o68PxY+eyXoKiSq\ndm1BW/ZKFIQr2JucA3kdg65CoiqvI5x3SdBViNRZuIL98tviv0RE5Lg0oCgiEjEKdhGRiKlTsJvZ\nT8xsiZnVmFlJqooSEZHaq2uP/TPgRmB2CmoREZEUqNOHp+6+DMC0aZKIyGlDY+wiIhFzwh67mb0N\ntDrGX41299dO9kZmNhQYClBYWHjSBYqIyKk5YbC7e59U3MjdJwOTAUpKSjwV1xQRkaMFskCpvLy8\n0szWBXHvOmoJVAZdRIbpmaOvvj0vhPeZ257MF5l77TvPZjYAeBLIA6qAhe7et9YXPM2Z2cfuXq+m\ndeqZo6++PS9E/5nrOitmBjAjRbWIiEgKaFaMiEjEKNhPzeSgCwiAnjn66tvzQsSfuU5j7CIicvpR\nj11EJGIU7LVgZiPNzM2sZdC1pJuZjTWz5Wa22MxmmFnzoGtKFzPrZ2YrzGy1mT0QdD3pZmZtzOw9\nM1uW2MzvnqBryhQzi5nZJ2b2j6BrSQcF+ykyszbANcD6oGvJkLeAYne/GFgJjAq4nrQwsxgwEfgh\n0AkYZGadgq0q7fYDv3b3i4ArgbvrwTMn3QMsC7qIdFGwn7pxwG+AevHhhLu/6e77Ey//A7QOsp40\n6gasdvfP3X0v8CJwQ8A1pZW7b3L3BYk/f0U86AqCrSr9zKw1cB3wbNC1pIuC/RSYWX9gg7svCrqW\ngPwceCPoItKkAPjikNcV1IOQSzKz84HLgHnBVpIR44l3zmqCLiRdwnXmaQZ816ZnwIPADzJbUfqd\nzEZvZjaa+I/uUzNZWwYda+/pevFTmZk1AaYDI9x9Z9D1pJOZlQH/c/dyM+sVdD3pomA/wvE2PTOz\n7wHtgEWJ/edbAwvMrJu7b85giSl3oo3ezOw2oAy42qM7P7YCaHPI69bAxoBqyRgzyyYe6lPd/W9B\n15MBpUB/M7sWOANoamYvuPutAdeVUprHXktmthYocfcwbiR00sysH/An4Cp3/zLoetLFzBoQ/3D4\namADMB8Y7O5LAi0sjSzeQ5kCbHP3EUHXk2mJHvtIdy8LupZU0xi7nMhTwFnAW2a20Mz+HHRB6ZD4\ngHgYMIv4h4gvRznUE0qBnwK9E/+2CxM9WQk59dhFRCJGPXYRkYhRsIuIRIyCXUQkYhTsIiIRo2AX\nEYkYBbuISMQo2EVEIkbBLiISMf8HB7s8CFNBUpYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x110ebe198>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = gluon.loss.Quantile(tau=0.2)\n",
    "with ag.record():    # start recording\n",
    "    theloss = loss(output, thelabel)\n",
    "theloss.backward()   # and compute the gradient\n",
    "\n",
    "plt.plot(output.asnumpy(), theloss.asnumpy(), label='Quantile loss 0.2')\n",
    "plt.plot(output.asnumpy(), output.grad.asnumpy(), label='Gradient of Quantile loss 0.2')\n",
    "\n",
    "# and now for the same loss function with tau = 0.6 \n",
    "loss = gluon.loss.Quantile(tau=0.6)\n",
    "\n",
    "with ag.record():    # start recording\n",
    "    theloss = loss(output, thelabel)\n",
    "theloss.backward()   # and compute the gradient\n",
    "\n",
    "plt.plot(output.asnumpy(), theloss.asnumpy(), label='Quantile loss 0.6')\n",
    "plt.plot(output.asnumpy(), output.grad.asnumpy(), label='Gradient of Quantile loss 0.6')\n",
    "\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## $\\epsilon$ Insensitive Loss\n",
    "\n",
    "In some cases we do not care about small deviations from the truth. More to the point, we do not care about deviations up to $\\epsilon$. Beyond that, we might care in a linear fashion. For instance, a screw might have a tolerance of $\\epsilon$ and the work to make anything fit beyond that would be linear in the diameter of the screw (yes, it's a contrived example). The associated loss function (desribed in detail by a paper by Vapnik, Golovich and Smola, 1995) is given by:\n",
    "\n",
    "$$l(y,f) = \\mathrm{max}(0, |y-f| - \\epsilon)$$\n",
    "\n",
    "As you can see, it contains a region $[y-\\epsilon, y+\\epsilon]$ where the derivative vanishes. Outside that range it is constant. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8Tdf6x/HPyixBSEQNQaQqhiQSOUFp0aqhNRRVM0lR\n2mq1euty61bnedCrlCoaU9FSwy2lpVpDXZIQQ8wi5iExhCQyr98fOD80SOSc7OSc5/16eZE4Z+3v\nzvBknbVXnq201gghhLAdDkYHEEIIYVlS2IUQwsZIYRdCCBsjhV0IIWyMFHYhhLAxUtiFEMLGSGEX\nQggbI4VdCCFsjBR2IYSwMU5GHLRSpUraz8/PiEMLIUSpFRsbm6y19rnb4wwp7H5+fsTExBhxaCGE\nKLWUUkcK8jhZihFCCBsjhV0IIWyMFHYhhLAxhqyx25rs7GyOHz9ORkaG0VGEENe4ubnh6+uLs7Oz\n0VGKnRR2Czh+/DjlypXDz88PpZTRcYSwe1przp07x/Hjx6ldu7bRcYqdLMVYQEZGBt7e3lLUhSgh\nlFJ4e3vb7atoKewWIkVdiJLFnr8nS1Vh33b0AlP+PGR0DCGEuCdxxy4Wy3FKVWFfvO0EH/2yl4Wx\nx42OUuI4OjoSEhJi/vPRRx8VeoyYmBhGjBgBQFRUFC+++KJFsk2ZMoVZs2ZZZKx7ceN5/fHHH/z1\n11/m/7NUtrfeeovPPvusyOPkZ+XKlQQEBFCnTp3bfl6joqLw8fExf/6nTZtmlSzi3i2MPU7XSRv5\necdJqx+rVF08faNTAw4lpfKvn3ZQo2IZmvp7Gx2pxChTpgxxcXFFGsNkMmEymSyU6P8999xzFh+z\nMG48rz/++IOyZcvSvHlzwPhsd5Obm8vw4cP57bff8PX1JTw8nC5dutCgQYO/PbZXr15MnDjRgJTi\nbjYnnONfP+2gRR1v2jesYvXjlaoZu7OjA1/3DaOGlzvD5sSSmJxmdKQSz8/Pj9GjR9OkSROaNGnC\nwYMHAfjxxx8JDAykUaNGtGzZErha9Dp16vS3MY4cOUKbNm0IDg6mTZs2HD16FIDIyEhGjBhB8+bN\n8ff3Z+HChflmuHE227p1a3OeunXrsn79egDi4+Np0qQJISEhBAcHc+DAAQDmzJljfv+wYcPIzc0F\noGzZsowdO5ZGjRrRrFkzzpw5c9fzSkxMZMqUKYwfP56QkBDWr19vzrZnzx6aNGlizpyYmEhwcDAA\nsbGxtGrVirCwMNq3b8+pU6fu+DGPi4ujWbNmBAcH061bNy5cuADAhAkTaNCgAcHBwfTu3RuAP//8\n0zzLDg0N5fLlyzeNtWXLFurUqYO/vz8uLi707t2bpUuX3vH4omRJTE5j2JxYani583XfMJwdrV92\nS9WMHcDT3ZnvIsPpOmkjg6KiWfxCCzzdS84+1bf/G8/uk5csOmaDauV5s3PDOz7mypUrhISEmN/+\n17/+Ra9evQAoX748W7ZsYdasWbzyyiv8/PPPvPPOO6xatYrq1atz8eKd1/1efPFFBg4cSEREBDNm\nzGDEiBEsWbIEgFOnTrFhwwb27t1Lly5d6NGjx13PJycnhy1btrBixQrefvttVq9ezZQpU3j55Zfp\n168fWVlZ5ObmsmfPHhYsWMDGjRtxdnbmhRdeYO7cuQwcOJC0tDSaNWvG+++/zz//+U++/fZb/v3v\nf9/xvPz8/HjuuecoW7Ysr732GgBr1qwBoH79+mRlZZGQkIC/vz8LFiygZ8+eZGdn89JLL7F06VJ8\nfHxYsGABY8eOZcaMGbc9v4EDB/LVV1/RqlUrxo0bx9tvv82XX37JRx99xOHDh3F1dTVn++yzz5g0\naRItWrQgNTUVNze3m8Y6ceIENWrUML/t6+vL5s2b8z3uokWLWLduHXXr1mX8+PE3PU8YIyU9m0FR\n0Sjgu8jwYqtVpWrGfl0tbw++GWDi2IV0np8bS3ZuntGRDHd9Keb6n+tFHaBPnz7mvzdt2gRAixYt\niIyM5NtvvzXPgm9n06ZN9O3bF4ABAwawYcMG8/917doVBwcHGjRoYJ4130337t0BCAsLIzExEYAH\nH3yQDz74gI8//pgjR45QpkwZ1qxZQ2xsLOHh4YSEhLBmzRoSEhIAcHFxMb+6uHGcwpzXrXr27MkP\nP/wAwIIFC+jVqxf79u1j165dtG3blpCQEN577z2OH7/9NZ6UlBQuXrxIq1atAIiIiGDdunUABAcH\n069fP+bMmYOTk5M576uvvsqECRO4ePGi+f3Xaa3/doz8dnt07tyZxMREduzYwWOPPUZEREShzl1Y\nXnZuHs/PjeXYhXS+GWCilrdHsR271M3Yr2tS24uPugfzjx+388aSXXzYPahEbG+628zaCDd+XK7/\ne8qUKWzevJnly5cTEhJSqPX5G8dzdXU1//t6ERo7dizLly8HyHfc689xdHQkJycHgL59+9K0aVOW\nL19O+/btmTZtGlprIiIi+PDDD/82hrOzsznHjeMU5bx69erF008/Tffu3VFK8cADD7Bz504aNmxo\n/oFYFMuXL2fdunUsW7aMd999l/j4eMaMGUPHjh1ZsWIFzZo1Y/Xq1dSrV8/8HF9fX44dO2Z++/jx\n41SrVu1vY3t7///1pmeffZbRo0cXOa+4d1pr3liyi78OnePzpxvRpLZXsR6/VM7Yr3sqzJcXH6nD\n/OhjfLs+weg4JdaCBQvMfz/44IMAHDp0iKZNm/LOO+9QqVKlm4rHrZo3b878+fMBmDt3Lg899NAd\nj/f++++bXzkU1PUlkBEjRtClSxd27NhBmzZtWLhwIWfPngXg/PnzHDly566ldzuvcuXK/W0d+7r7\n778fR0dH3n33XfMrnoCAAJKSksyFPTs7m/j4+Nse39PTk4oVK5qvHcyePZtWrVqRl5fHsWPHeOSR\nR/jkk0+4ePEiqampHDp0iKCgIEaPHo3JZGLv3r03jRceHs6BAwc4fPgwWVlZzJ8/ny5duvztuDeu\n+y9btoz69evf8eMkrOvb9QnMjz7Gi4/U4akw32I/fqmdsV/3atu6HE5O48Nf9lLL26NYrjiXRLeu\nsXfo0MG8NS4zM5OmTZuSl5fHvHnzABg1ahQHDhxAa02bNm1o1KgRf/75Z75jT5gwgUGDBvHpp5/i\n4+PDd999Z/H8CxYsYM6cOTg7O1OlShXGjRuHl5cX7733Hu3atSMvLw9nZ2cmTZpErVq1bjvO3c6r\nc+fO9OjRg6VLl/LVV1/97fm9evVi1KhRHD58GLi65LNw4UJGjBhBSkoKOTk5vPLKKzRsePtXZjNn\nzuS5554jPT0df39/vvvuO3Jzc+nfvz8pKSlorRk5ciQVKlTgjTfeYO3atTg6OtKgQQMef/zxm8Zy\ncnJi4sSJtG/fntzcXAYNGmQ+9rhx4zCZTHTp0oUJEyawbNkynJyc8PLyIioqqjAffmFBq+JP8+Ev\ne+kYVJVX29Y1JIPKbw3P2kwmk7bkjTYysnPpNfV/7D99mR+fe5DA6p4WG7sg9uzZU2JnSNdvalKp\nUiWjowhR7Ir7e3PXiRSenrKJulXKsWBoM9ycHS06vlIqVmt91z3JpXop5jo3Z0e+HRiGl4cLg2dG\nczrFPvtDCCGMczolg8Ezo/HycOHbgWEWL+qFYROFHaByOTemRZhIzchh8Mxo0rNyjI5UIiQmJsps\nXQgrS8+6WndSM3KYFmGicjm3uz/JimymsAPUr1qeiX0bs+fUJV6ZH0deXvEvMwkh7EtenuaV+XHs\nOXWJiX0bU79qeaMj2VZhB3ikXmXe6NSAX3ef4eOVe+/+BCGEKIKPV+7l191neKNTAx6pV9noOIAN\n7IrJT2RzPxKS0vhmXQK1K3nQu0lNoyMJIWzQ/C1H+WZdAgOa1SKyuZ/RccxssrArpXizcwOOnE/n\n30t2UdPLneZ1ZJ1ZCGE5fx1M5t9LdtGyrg9vdm5QIn5B8jqLLcUopRyVUtuUUj9basyicHJ0YGLf\nUGpX8uC5ObEcSko1OpJVnTlzhr59++Lv709YWBgPPvggixcvLtKYNzbvGjduHKtXr76nceLi4lix\nYkWhn9enTx+Cg4MZP37833JVr179pjbFd+t3k58hQ4awe/du4Oq20OTk5EKPkZ/rnSONcuN5ffDB\nBzf9n6WylS1b1iLj3CozM5NevXpRp04dmjZtam4VcSs/Pz+CgoIICQmxSkfSuzmUlMpzc2KpXcmD\niX1DcSqGxl6ForW2yB/gVeB74Oe7PTYsLEwXl6Pn0nTjd37VLT/5XZ9PzbTKMXbv3m2VcQsqLy9P\nN2vWTE+ePNn8vsTERD1hwoS/PTY7O7vA47755pv6008/LXK+7777Tg8fPrxQzzl16pSuWbOmVXPd\nqFatWjopKcmiY5YEHh4epWrcSZMm6WHDhmmttZ43b57u2bNnvo8r6OfLGt+b51MzdctPfteN3/lV\nHz2XZvHx7wSI0QWoxxb5MaOU8gU6AiWuu38NL3emDjRxKiWDYXNiycwpXGOo0uD333/HxcXlpt7i\ntWrV4qWXXgKu3oTh6aefpnPnzrRr147U1FTatGlD48aNCQoKuqkN7Pvvv09AQACPPfYY+/btM78/\nMjLS3Jb3dm1s82vJm5WVxbhx41iwYAEhISHm9gbXZWRk8MwzzxAUFERoaChr164FoF27dpw9e9bc\nXrcgoqKiePLJJ+nQoQMBAQG8/fbbAKSlpdGxY0caNWpEYGCgOUPr1q3J7xflvvjiCwIDAwkMDOTL\nL78Erm4brV+/Ps8++ywNGzakXbt2XLlyJd8c12ezf/zxB61bt6ZHjx7Uq1ePfv36mfvpjBkzxtzC\n93qnyaSkJJ566inCw8MJDw9n48aNwNVXKIMGDaJ169b4+/szYcKEAp3XmDFjzL+R3K9fv5uy9erV\n66ZXUZGRkSxatIjc3FxGjRpFeHg4wcHBfPPNN3f8mGutGTVqFIGBgQQFBZkznDp1ipYtWxISEkJg\nYCDr168nNzeXyMhI82NvfSUGsHTpUnMDsx49erBmzZp8G6EZJTMnl2FzYjmVksHUgSZqeLkbHSlf\nllpj/xL4J1Dudg9QSg0FhgLUrFm8FzPDalXk0x7BvDw/jtd/2sVnTwdbbz3slzFweqdlx6wSBI/f\n/o5I8fHxNG7c+I5DbNq0iR07duDl5UVOTg6LFy+mfPnyJCcn06xZM7p06cLWrVuZP38+27ZtIycn\nh8aNGxMWFnbTOHdrY5tfS9533nmHmJiYfG8CMWnSJAB27tzJ3r17adeuHfv372fZsmV06tTptv1m\nxo8fz5w5cwCoWLGi+QfCli1b2LVrF+7u7oSHh9OxY0eOHDlCtWrVzI3JUlJSbvtxio2N5bvvvmPz\n5s1orWnatCmtWrWiYsWKHDhwgHnz5vHtt9/Ss2dPFi1aRP/+/e/4cd+2bRvx8fFUq1aNFi1asHHj\nRho0aMDixYvZu3cvSinzMtLLL7/MyJEjeeihhzh69Cjt27dnz549AOzdu5e1a9dy+fJlAgICeP75\n51m5cuUdz+ujjz5i4sSJ+X4Me/fuzYIFC3jiiSfIyspizZo1TJ48menTp+Pp6Ul0dDSZmZm0aNGC\ndu3aUbt27XzP76effiIuLo7t27eTnJxMeHg4LVu25Pvvv6d9+/aMHTuW3Nxc0tPTiYuL48SJE+za\ntQsg3+WzG9sUOzk54enpyblz5/72uxhKKdq1a4dSimHDhjF06NA7fh4sQWvN6z/tYsvh8/yndwhh\ntSpa/Zj3qsiFXSnVCTirtY5VSrW+3eO01lOBqXC1pUBRj1tYT4ZU53ByGl+uPoC/jwfDH6lT3BGK\nzfDhw9mwYQMuLi5ER0cD0LZtW7y8rnaY01rz+uuvs27dOhwcHDhx4gRnzpxh/fr1dOvWDXf3q7OQ\n/JpN3djGFq7e4adq1arm/8+vJe+dbNiwwfzKol69etSqVYv9+/dTvvyd9wKPHDnSPNO9Udu2bc2d\nDrt3786GDRt44okneO211xg9ejSdOnXi4YcfvmOebt264eHhYR5j/fr1dOnShdq1a5v78RT0/Jo0\naYKv79UmUCEhISQmJtKsWTPc3NwYMmQIHTt2NLcfXr16tXltHODSpUvmhmUdO3bE1dUVV1dXKleu\nzJkzZwgKCirwed3q8ccfZ8SIEWRmZrJy5UpatmxJmTJl+PXXX9mxY4f51VlKSgoHDhy4bWHfsGED\nffr0wdHRkfvuu49WrVoRHR1NeHg4gwYNIjs7m65duxISEoK/vz8JCQm89NJLdOzYkXbt2v1tvPxm\n5/lNwjZu3Ei1atU4e/Ysbdu2pV69euYbq1jL138cYtHW47zy2AM8GVLdqscqKkvM2FsAXZRSTwBu\nQHml1Byt9Z2nMgZ4uc0DHE5O49NV+6hdyYMngqre/UmFdYeZtbU0bNiQRYsWmd+eNGkSycnJN11U\nul6o4GqHxqSkJGJjY3F2dsbPz4+MjKttGO72SkZrfcc2tvm15L3beJZ0a36lFHXr1iU2NpYVK1bw\nr3/9i3bt2jFu3LhC57mxRbGjoyNXrlzh2LFjdO7cGbh6m71bb7V363NycnJwcnJiy5YtrFmzhvnz\n5zNx4kR+//138vLy2LRpE2XKlLnrsXNycgp1Xrdyc3OjdevWrFq1igULFph79mut+eqrr2jfvn2B\nxrndx6tly5asW7eO5cuXM2DAAEaNGsXAgQPZvn07q1atYtKkSfzwww9/u2HJ9TbFvr6+5OTkkJKS\nYp6Q3Oh66+LKlSvTrVs3tmzZYtXCvmLnKT5dtY8nQ6rxcpsHrHYcSynyGrvW+l9aa1+ttR/QG/i9\nJBZ1uPpN/vFTwYTVqsjIBXFsL6Y7hlvbo48+SkZGBpMnTza/Lz09/baPT0lJoXLlyjg7O7N27Vpz\nK9yWLVuyePFirly5wuXLl/nvf//7t+cWto0t3LlVbsuWLZk7dy4A+/fv5+jRowQEBNz5hO/gt99+\n4/z581y5coUlS5bQokULTp48ibu7O/379+e1115j69att31+y5YtWbJkCenp6aSlpbF48eI7zoRr\n1KhhblFc0PunpqamkpKSwhNPPMGXX35pXipp167dTctVd2t7XJDzcnZ2Jjs7O9/n9+7dm++++471\n69ebC3n79u2ZPHmy+Tn79+8nLe32t6Bs2bIlCxYsIDc3l6SkJNatW0eTJk04cuQIlStX5tlnn2Xw\n4MFs3bqV5ORk8vLyeOqpp3j33XfzzdulSxdmzpwJwMKFC3n00Uf/9sM6LS3N/PWUlpbGr7/+SmBg\n4B0/VkWx/dhFRi6II6xWRT5+yorLuBZkk/vY78TN2ZFvBoTRddJGhsyKYcnwFlSv8PcZUmmilGLJ\nkiWMHDmSTz75BB8fHzw8PPj444/zfXy/fv3o3LkzJpOJkJAQ840dGjduTK9evQgJCaFWrVr5FrR7\naWP7yCOP8NFHHxESEnLTLfsAXnjhBZ577jmCgoJwcnIiKirqptnp7dy4xg6Yb9X30EMPMWDAAA4e\nPEjfvn0xmUysWrWKUaNG4eDggLOz800/AG/VuHFjIiMjzfc/HTJkCKGhoQVadimoy5cv8+STT5KR\nkYHW2nwRccKECQwfPpzg4GBycnJo2bIlU6ZMue04O3fuvOt5DR06lODgYBo3bmz+AXpdu3btGDhw\nIF26dMHFxcV8vomJiTRu3BitNT4+PuaPbX66devGpk2baNSoEUopPvnkE6pUqcLMmTP59NNPcXZ2\npmzZssyaNYsTJ07wzDPPkJd39Y5n+d1AZfDgwQwYMIA6derg5eVlvg/AyZMnGTJkCCtWrODMmTN0\n69YNuHpNp2/fvnTo0OFOH/J7duLiFYbMisGnnCvfDDC2sVdh2ETb3ntx4Mxlun/9F9UrlmHh880p\n63rvP+NKctteexIVFXXbi7TCPhXlezM1M4cek//ixIUr/PRCcx6477Z7Q4qNXbXtvRcP3FeOSf0a\nc+BsKiPmbSNXGoYJIa7JzdOMmLeNA2dTmdSvcYko6oVht4UdoGVdH97q0pDf957l/eV7jI4jiigy\nMlJm68Ii3l++h9/3nuWtLg1pWdfH6DiFZndr7Lca0KwWCUmpzNh4GH8fD/o3u/1t1+5Ea10qLqoI\nYS/udZl5zv+OMGPjYZ5p4ceAe6wHRrPrGft1/+7YgEfrVebNZfGs259U6Oe7ublx7ty5EvUbckLY\nM601586dw82tcDe8WLc/iTeXxfNovcr8u2MDK6WzPru9eHqrolwoyc7O5vjx4+a94EII47m5ueHr\n64uzs3OBHm/JDRXWUtCLp1LYb3Di4hWenLiRMi4OLHmhBd5l777tTghR+p1LzaTr1xu5kpXH0hdL\n7hZo2RVzD6pXKMO0CBNnL2UydHYsGdm21zBMCHGzjOxchs6O5eylTKZFmEpsUS8MKey3CKlRgfG9\nQog9coHRi3bIurkQNkxrzehFO4g9coHxvUIIqVHB6EgWIYU9H08EVWVU+wCWxp1kwpqDRscRQljJ\nhDUHWRp3klHtA6zTO8ogJe/qQAnxQuv7SUhKY/zq/dT28aBLo2pGRxJCWNCy7ScZv3o/TzX25YXW\n9xsdx6Jkxn4bSik+6B5IEz8vXvtxO7FHLhgdSQhhIbFHLvDaj9tp4ufFB90Dbe53UKSw34GrkyNT\nBoRR1dONobNiOHb+9h0ThRClw7Hz6QydFUNVTzemDAjD1al0NPYqDCnsd+Hl4cKMyHCyc/MYPDOa\nSxn5t0AVQpR8lzKyGTwzmuzcPGZEhuPl4WJ0JKuQwl4A9/uUZUr/MBKS0njx+23k5OYZHUkIUUg5\nuXm8+P02EpLSmNI/jPt9yhodyWqksBdQ8zqVeK9rIOv2J/HOz7vv/gQhRInyzs+7Wbc/ife6BtK8\nTqW7P6EUk10xhdC7SU0SktOYui4B/0oeRLbI/z6QQoiSJWrjYWZtOsLQlv70blLT6DhWJ4W9kEZ3\nqMfh5DTe+Xk3tbw9eKReZaMjCSHuYO3es7zz827aNriP0R3qGR2nWMhSTCE5Oij+0zuE+lXL89K8\nbew9fcnoSEKI29h7+hIvzdtG/arl+U/vEBwdbGtb4+1IYb8H7i5OTI8Ix8PVkcFRMZy9LF0dhShp\nzl7OYHBUDB6ujkyPCMfdxX4WKKSw36Mqnm5MjwjnfFoWz86ShmFClCQZ2bk8OyuW82lZTI8Ip4pn\n4fqyl3ZS2IsgsLonX/YOYcfxi/zjh+3kyX1ThTBcXp7mHz9sZ8fxi3zZO4TA6p5GRyp2UtiLqH3D\nKozpUI/lO0/xxW/7jY4jhN374rf9LN95ijEd6tG+YRWj4xjCfhadrGhoS38SktKYuPYgtSt58FSY\nr9GRhLBLi2KPM3HtQXqZajC0pb/RcQwjM3YLUErxbtdAHvT3ZsxPO9hy+LzRkYSwO1sOn2fMTzt4\n0N+bd7vaXmOvwpDCbiEuTg5M6R9GjYruDJsdQ2JymtGRhLAbiclpDJsdQ42K7kzpH4aLk32XNvs+\newvzdHdmRmQ4Ghg0M5qUdGkYJoS1paRnM2hmNBqYERmOp3vBbl5ty4pc2JVSbkqpLUqp7UqpeKXU\n25YIVlr5VfLgm/5hHDufzvNzY8mWhmFCWE12bh7Pz43l2Pl0vukfhl8lD6MjlQiWmLFnAo9qrRsB\nIUAHpVQzC4xbajX19+bD7sH8degc45bukvumCmEFWmvGLd3FX4fO8WH3YJr6exsdqcQo8q4YfbVq\npV570/naH7uvZD3CfElISuXrPw7hX6ksz9rxFXohrGHa+sPM23KMF1rfTw/ZiXYTi6yxK6UclVJx\nwFngN631ZkuMW9q91i6AJ4Kq8MEve/g1/rTRcYSwGb/Gn+aDX/bwRFAVXmsXYHScEscihV1rnau1\nDgF8gSZKqcBbH6OUGqqUilFKxSQlJVnisCWeg4Pi86dDCK7uycvz49h1IsXoSEKUertOpPDy/DiC\nq3vy+dMhONhJY6/CsOiuGK31ReAPoEM+/zdVa23SWpt8fHwsedgSrYyLI98ONFHR3ZkhM2M4nSIN\nw4S4V6dTMhgyM4aK7s58O9BEGRfbu1+pJVhiV4yPUqrCtX+XAR4D9hZ1XFtSubwb0yLCuZyRzZBZ\n0aRn5RgdSYhSJz0rhyGzormckc20iHAql7evxl6FYYkZe1VgrVJqBxDN1TX2ny0wrk1pUK08E/qE\nsvvkJV6ZHycNw4QohLw8zSvz49h98hIT+oTSoFp5oyOVaEUu7FrrHVrrUK11sNY6UGv9jiWC2aI2\n9e9jbMcG/Lr7DB+vkhc1QhTUx6v28uvuM4zt2IA29e8zOk6JJ03AitmgFn4kJKXyzZ9X75vaK9z2\n778oRFEsiD7KN38m0K9pTQa18DM6Tqkghb2YKaV4q0tDjp5PZ+ziXdTwcqf5/bZ9x3Qh7tVfh5IZ\nu3gXDz9Qibe6NLTrxl6FIb1iDODs6MDEvo3xq+TB83O2kpCUevcnCWFnEpJSeX7OVvwqeTCxb2Oc\nHaVcFZR8pAziWcaZGRHhODooBkVFcyEty+hIQpQYF9KyGBQVjaODYkZEOJ5lpLFXYUhhN1BNb3em\nDgjj5MUMnpsTS1aONAwTIisnj+fmxHLyYgZTB4RR09vd6EiljhR2g5n8vPikRzCbD5/n9cU7pWGY\nsGtaa15fvJPNh8/zSY9gTH5eRkcqleTiaQnQNbQ6CclpTFhzAH8fD15oXcfoSEIYYvKfh1gYe5wR\nbR6ga2h1o+OUWlLYS4iRjz3A4eQ0Plm5j9reHjweVNXoSEIUq192nuKTlfvo3KgaIx97wOg4pZos\nxZQQSik+7RFMaM0KjPwhjh3HLxodSYhis+P4RUb+EEdozQp82iNYtjUWkRT2EsTN2ZGpA0x4e7gy\neGYMJy9eMTqSEFZ38uIVBs+MwdvDlakDTLg5S2OvopLCXsL4lHNlRmQ4V7JyGTwzhrRMaRgmbFda\nZg6DZ8ZwJSuXGZHh+JRzNTqSTZDCXgIFVCnHxL6h7Dt9iRHztpErDcOEDcrN04yYt419py8xsW8o\nAVXKGR3JZkhhL6FaB1TmrS4NWbP3LB+s2GN0HCEs7oMVe1iz9yxvdWlI64DKRsexKbIrpgQb+KAf\nCUlpTN+QYQQlAAAW5UlEQVRwGH8fD/o1rWV0JCEsYu7mI0zfcJjI5n4MfNDP6Dg2Rwp7CffvjvVJ\nPJfGuKXx1PRy5+EH7OfuU8I2rT+QxLil8bQO8OHfHesbHccmyVJMCefk6MBXfUKp41OWF+Zu5eDZ\ny0ZHEuKeHTx7mRfmbqWOT1m+6hOKkzT2sgr5qJYC5dycmR5pwtXJkWeiojmXmml0JCEK7VxqJs9E\nRePq5Mj0SBPl3KSxl7VIYS8lfCu68+3AMM5eymTY7Fgyc3KNjiREgWXm5DJsdixnL2Xy7cAwfCtK\nYy9rksJeioTWrMjnPRsRc+QCYxZJwzBROmitGbNoJzFHLvB5z0aE1qxodCSbJxdPS5lOwdVITE7j\ns1/3U7uSByPaSE8NUbJ99ftBFm87wWvt6tIpuJrRceyCFPZSaPgjdUhITuOL364W986N5JtFlEz/\n3X6SL37bT/fG1Rn+iHQtLS6yFFMKKaX4sHsQTfy8+MeP29l69ILRkYT4m61HL/CPH7fTxM+LD7sH\nSWOvYiSFvZRydXJkyoAwqnq6MXRWDMfOpxsdSQizY+fTGTorhqqebkwZEIarkzT2Kk5S2EsxLw8X\npkeEk5WTx5CZMVzOyDY6khBczshmyMwYsnLymB4RjpeHi9GR7I4U9lKuTuWyTO4fxqGkVF78fhs5\nuXLfVGGcnNw8Xvx+G4eSUpncP4w6lcsaHckuSWG3AS3qVOLdroH8uT+Jd3/ebXQcYcfe/Xn31a/D\nroG0qFPJ6Dh2S3bF2Ig+TWqSkJTKt+sP4+9TlojmfkZHEnZm5l+JzNx0hGcfrk2fJjWNjmPXijxj\nV0rVUEqtVUrtUUrFK6VetkQwUXhjHq/PY/Xv4+3/xrN231mj4wg7snbfWd7+bzyP1b+PMY9LYy+j\nWWIpJgf4h9a6PtAMGK6UamCBcUUhOToo/tM7hHpVyvPS99vYd1oahgnr23f6Mi99v416Vcrzn94h\nODrItkajFbmwa61Paa23Xvv3ZWAPUL2o44p74+HqxPRIEx6ujgyKiibpsjQME9aTdDmTQVHReLg6\nXvu6k9XdksCiF0+VUn5AKLDZkuOKwqnqWYZpA8M5n5bFs7NiyMiWhmHC8jKyc3l2Vgzn07KYNjCc\nqp5ljI4krrFYYVdKlQUWAa9orS/l8/9DlVIxSqmYpKQkSx1W3EaQryfje4Ww/fhFXvtxO3ly31Rh\nQXl5mtd+3M724xcZ3yuEIF9PoyOJG1iksCulnLla1OdqrX/K7zFa66laa5PW2uTjI3cBKg4dAqsw\nukM9ft5xii9X7zc6jrAhX67ez887TjG6Qz06BFYxOo64RZEXxNTVBhDTgT1a6y+KHklY0rCW/iQk\npTLh94PU9vGgW6iv0ZFEKbd423Em/H6QniZfhrX0NzqOyIclZuwtgAHAo0qpuGt/nrDAuMIClFK8\n1zWIB/29Gb1wJ9GJ542OJEqx6MTzjF64kwf9vXmvqzT2KqkssStmg9Zaaa2DtdYh1/6ssEQ4YRku\nTg5M7t8Y34plGDY7lqPnpGGYKLyj59IZNjsW34plmNy/MS5O8ovrJZV8ZuxEBXcXpkeGk6c1z0Rt\nIeWKNAwTBZdyJZtnoraQpzXTI8Op4C6NvUoyKex2pHYlD6b0D+Po+XSGz91KtjQMEwWQnZvH8Llb\nOXo+nSn9w6hdycPoSOIupLDbmWb+3nzQLYgNB5N5c1m83DdV3JHWmjeXxbPhYDIfdAuimb+30ZFE\nAcividmhp001SEhOY/Ifh/Cv5MGQh2Vng8jf9A2H+X7zUZ5vfT9Pm2oYHUcUkBR2OzWqXQCJyWm8\nv2IPft4ePNbgPqMjiRJm9e4zvL9iD48HVmFUuwCj44hCkKUYO+XgoPiiZwhB1T0ZMX8b8SdTjI4k\nSpD4kymMmL+NoOqefNEzBAdp7FWqSGG3Y2VcHJk20IRnGWeGzIzh7KUMoyOJEuDspQyGzIzBs4wz\n0waaKOMi9ystbaSw27nK5d2YHhFOypVshsyK4UqWNAyzZ1eychkyK4aUK9lMjwincnk3oyOJeyCF\nXdCgWnkm9A5l54kURi6Ik4ZhdiovTzNyQRw7T6QwoXcoDaqVNzqSuEdS2AUAjzW4j7FP1Gdl/Gk+\n/XWf0XGEAT79dR8r408z9on6cjG9lJNdMcJs8EO1zdsga1fyoKdsb7MbP8QcY/Ifh+jbtCaDH6pt\ndBxRRFLYhZlSire7NOTY+XTGLt5JTS93+YUUO/C/hHOMXbyThx+oxNtdGkpjLxsgSzHiJs6ODkzs\n25ha3h4Mmx1LQlKq0ZGEFSUkpTJsdiy1vD2Y2Lcxzo5SEmyBfBbF33iWcWZGRDiODorBM2O4mJ5l\ndCRhBRfTsxg8MwZHB8WMiHA8yzgbHUlYiBR2ka+a3u5MHRDGiQtXGDY7lqwcaRhmS7Jy8hg2O5YT\nF64wdUAYNb3djY4kLEgKu7gtk58Xn/QIZvPh84xdvFMahtkIrTVjF+9k8+HzfNIjGJOfl9GRhIXJ\nxVNxR11Dq5tvrefvU5bnW99vdCRRRFP+TODH2OOMeLQOXUOrGx1HWIEUdnFXI9vWJSE5jY9X7qV2\nJXc6BFY1OpK4Ryt3neLjlXvpFFyVkW3rGh1HWIksxYi7Ukrx2dONCK1ZgVcWxLHj+EWjI4l7sOP4\nRV5ZEEdozQp89nQj2dZow6SwiwJxc3Zk6gAT3h6uDJkZw6mUK0ZHEoVwKuUKQ2bG4O3hytQBJtyc\npbGXLZPCLgrMp5wrMyLDSc/KZXBUDGmZOUZHEgWQlpnD4KgY0rNymREZjk85V6MjCSuTwi4KJaBK\nOSb2DWXv6Uu8PH8budIwrETLzdO8PH8be09fYmLfUAKqlDM6kigGUthFobUOqMybnRuyes9ZPlyx\nx+g44g4+XLGH1XvO8mbnhrQOqGx0HFFMZFeMuCcRzf1ISEpl2obD+PuUpW/TmkZHErf4fvNRpm04\nTMSDtYho7md0HFGMpLCLe/ZGpwYcOZ/OG0t3UdPLnYceqGR0JHHNhgPJvLF0F60DfHijUwOj44hi\nJksx4p45OTrwVZ9Q6viU5fm5sRw8e9noSAI4ePYyz8+NpY5PWb7qE4qTNPayO/IZF0VSzs2Z6ZEm\nXJ0cGBQVw/k0aRhmpPNpWQyKisHVyYHpkSbKuUljL3tkkcKulJqhlDqrlNplifFE6eJb0Z2pA02c\nvpTBsNkxZObIfVONkJmTy7DZMZy+lMHUgSZ8K0pjL3tlqRl7FNDBQmOJUqhxzYp8/nQjohMvMGaR\nNAwrblprxizaSXTiBT5/uhGNa1Y0OpIwkEUunmqt1yml/Cwxlii9OjeqxuHkNL74bT/+lTx4qc0D\nRkeyGxN/P8jibSd4tW1dOjeqZnQcYTDZFSMs6qVH63A4OY3Pf9tPbR8POgXbSZHRGlaOgXMHi/3Q\nZy9nEnTyEr94u1LvVHmYU+wRRGE8MhaqN7bqIYqtsCulhgJDAWrWlD3PtkopxUdPBXHsfDr/+GE7\n1SuUIdQelgVys2HzFChXDcoXX/fLtMwcTiWlUt3Vifs9HVFXLhTbscU9yrN+Kw5lqbXQa0sxP2ut\nA+/2WJPJpGNiYixyXFEynUvNpNvXf5GelcOS4S1s/0Jedga8fx+0GQcP/6NYDnn8QjpdJ23E3cWJ\nxS80x7us9ICxdUqpWK216W6Pk+2Owiq8y7oyI9JEZk4eg6NiuJyRbXQk69LXbh2oiudb6nJGNoOj\nYsjMyWNGpEmKuriJpbY7zgM2AQFKqeNKqcGWGFeUbnUql2NyvzAOJqXy0rxt5OTa8H1Ti7Gw5+Tm\n8dK8bRxMSmVyvzDqVJbGXuJmFvkq1Fr30VpX1Vo7a619tdbTLTGuKP0eeqAS7zzZkD/2JfHechtu\nGGYu7Nbvc/7e8j38sS+Jd55sKG0cRL5kV4ywun5Na5GQlMb0DYfx9/Fg4IN+RkeyvGKasc/alEjU\nX4kMfqg2/ZrWsuqxROklhV0Ui9efqM+Rc2m8tSyeml7uttdCthgK+x/7zvLWsngeq1+Z15+ob7Xj\niNJPLp6KYuHooPhP71ACqpTnxe+3se+0jTUMs3Jh33f6Mi9+v42AKuX5T+9QHB3kfqXi9qSwi2Lj\n4erE9AgT7i6ODIqKJulyptGRLMdc2C1fcJMuZzIoKhp3F0emR5jwcJUX2uLOpLCLYlWtQhmmRZg4\nl5bJ0NkxZGTbSMMwK83YM7JzGTo7hnNpmUyLMFGtQhmLji9skxR2UeyCfSvwZa8Qth29yKiFO2yj\nYdj1wu5guV0xWmtGLdzBtqMX+bJXCMG+FSw2trBtUtiFIToEVuWfHQL47/aTjF99wOg4RWeFGfv4\n1Qf47/aT/LNDAB0Ci69NgSj9ZLFOGOb5VvdzOCmNCWsO4F/Jg66h1Y2OdO/yri0pWaiwL9l2gglr\nDvB0mC/Pt7rfImMK+yEzdmEYpRTvdwuiaW0v/rlwBzGJ542OdO8sOGOPSTzPPxfuoGltL97vFoSy\nwgVZYduksAtDuTg5MKV/GNUrlmHo7FiOnks3OtK9sVBhP3ounaGzY6lesQxT+ofh4iTfoqLw5KtG\nGK6ihwvTI0zk5mkGzYwm5UopbBh2/QJwEVoKpFzJZtDMaHLzNNMjTFT0cLFQOGFvpLCLEsHfpyyT\n+zcmMTmNF7/fSnZpaxhWxH3s2bl5vPj9VhKT05jcvzH+PmUtGE7YGynsosRofn8lPugWxPoDyby1\nLL50bYMswlKM1pq3lsWz/kAyH3QLovn90thLFI3sihElSs/wGhxKTuWbPxPw9ynL4IdqGx2pYIpQ\n2GdsTGTu5qMMa+VPz/AaFg4m7JEUdlHijG5fj8TkNN5bvhs/b3fa1L/P6Eh3p+9tu+OaPWd4b/lu\n2je8j9Ht61khmLBHshQjShwHB8X4XiE0rFael+ZtY/fJS0ZHurt7+M3T3Scv8dK8bTSsVp7xvUJw\nkMZewkKksIsSyd3FiekR4ZR3c2bwzGjOXsowOtKdFXIp5uylDAbPjKa8mzPTI8Jxd5EXz8JypLCL\nEuu+8m5MizBxMT2bIbNiuJJVghuGFaKwX8nKZcisGC6mZzMtwsR95d2sHE7YGynsokQLrO7JhD6h\n7DyRwqs/xJGXV0J3ypj3sd/5WyovT/PqD3HsPJHChD6hBFb3LIZwwt5IYRclXtsG9/H64/X5Zddp\nPvt1n9Fx8lfAfeyf/bqPX3ad5vXH69O2QSm4KCxKJVnYE6XCkIdrk5Ccytd/HKJ2JQ+eNpWwbYEF\naAL2Y8wxvv7jEH2a1GDIw6VkG6colWTGLkoFpRTvPBlIizrevL54J/9LOGd0pJuZZ+z574r5X8I5\nXl+8kxZ1vHnnyUBp7CWsSgq7KDWcHR34um8YNbzceW5OLIeT04yO9P/ucPH0cHIaz82JpYaXO1/3\nDcPZUb7thHXJV5goVTzdnfkuMhwFDI6K5mJ6ltGRrrpNYb+YnsXgqGgU8F1kOJ7uzsWfTdgdKeyi\n1Knl7cE3A0wcu5DO83O2kpVTAhqG5VPYs3LyeH7OVo5dSOebASZqeXsYFE7YGynsolRqUtuLj7oH\nsynhHG8s2WV8w7BbCrvWmjeW7GJTwjk+6h5Mk9peBoYT9kZ2xYhS66kwXw4npzFx7UH8fTwYZuQt\n5G5pKTB1XQILYo7x4iN1eCrM17hcwi5ZZMaulOqglNqnlDqolBpjiTGFKIhX29alY1BVPlq5l1Xx\np40LcsM+9lXxp/lo5V46BlXl1bZ1jcsk7FaRC7tSyhGYBDwONAD6KKUaFHVcIQrCwUHxec9GBPtW\n4JX5cew6kWJMkGuF/WBSOq/MjyPYtwKf92wkjb2EISwxY28CHNRaJ2its4D5wJMWGFeIAnFzduTb\ngWF4ebgweGY0p1MMaBh2rbC//fMevDxc+HZgGG7O936bPCGKwhJr7NWBYze8fRxoaoFxhSiwyuWu\nNgwbMPl3Dn35OBcdrxTr8cvpS1QH0rLymDbYROVy0thLGMcShT2/15p/26KglBoKDAWoWbOmBQ4r\nxM3qVy3PtM7ehCzfylHlT4pDhWI79nnKcNy1Dv/s0on6VcsX23GFyI8lCvtx4MbGHb7AyVsfpLWe\nCkwFMJlMJbRFnyjtQnyvFtWaT70H9ToanEYIY1hijT0aeEApVVsp5QL0BpZZYFwhCq8I9x4VwlYU\necautc5RSr0IrAIcgRla6/giJxPiXkhhF8Iyv6CktV4BrLDEWEIUSd6duywKYQ9kWiNsSwFveCGE\nLZPCLmyLLMUIIYVd2Bgp7EJIYRc2Rgq7EFLYhY2Rwi6EFHZhY/S1m0o7yK4YYb+ksAvbIjN2IaSw\nCxtz/U5KUtiFHZOvfmFbZB+7EFLYhY2RpRghpLALG5N37eKpFHZhx+SrX9gWmbELIYVd2BgtTcCE\nkMIubIvM2IWQwi5sjBR2IaSwCxsjhV0IKezCxsg+diGksAsbIzN2IaSwCxtzvbBLEzBhx6SwC9si\nM3YhpLALGyOFXQgp7MLGSEsBIaSwCxsjM3YhpLALGyP92IWQwi5sjMzYhZDCLmyMFHYhpLALG6Pl\n4qkQRfrqV0o9rZSKV0rlKaVMlgolxD2TGbsQRZ6x7wK6A+sskEWIopPCLgRORXmy1noPgJKGS6Kk\nkMIuRNEKe7H781PYtdDoFKIkS0u++rcUdmHH7lrYlVKrgSr5/NdYrfXSgh5IKTUUGApQs2bNAge8\nSdnK4BNwb88V9sEnALzrgJOL0UmEMIzS13+hoyiDKPUH8JrWOqYgjzeZTDompkAPFUIIcY1SKlZr\nfdeNKvJ6VQghbExRtzt2U0odBx4EliulVlkmlhBCiHtV1F0xi4HFFsoihBDCAmQpRgghbIwUdiGE\nsDFS2IUQwsZIYRdCCBsjhV0IIWyMRX5BqdAHVSoJOFLsBy66SkCy0SGKmZyz7bO384XSe861tNY+\nd3uQIYW9tFJKxRTkt75siZyz7bO38wXbP2dZihFCCBsjhV0IIWyMFPbCmWp0AAPIOds+eztfsPFz\nljV2IYSwMTJjF0IIGyOF/R4opV5TSmmlVCWjs1ibUupTpdRepdQOpdRipVQFozNZi1Kqg1Jqn1Lq\noFJqjNF5rE0pVUMptVYptefaTelfNjpTcVFKOSqltimlfjY6izVIYS8kpVQNoC1w1OgsxeQ3IFBr\nHQzsB/5lcB6rUEo5ApOAx4EGQB+lVANjU1ldDvAPrXV9oBkw3A7O+bqXgT1Gh7AWKeyFNx74J2AX\nFye01r9qrXOuvfk/wNfIPFbUBDiotU7QWmcB84EnDc5kVVrrU1rrrdf+fZmrha66samsTynlC3QE\nphmdxVqksBeCUqoLcEJrvd3oLAYZBPxidAgrqQ4cu+Ht49hBkbtOKeUHhAKbjU1SLL7k6uQsz+gg\n1lKkG23YojvdvBt4HWhXvImsryA3LFdKjeXqS/e5xZmtGKl83mcXr8qUUmWBRcArWutLRuexJqVU\nJ+Cs1jpWKdXa6DzWIoX9Flrrx/J7v1IqCKgNbFdKwdUlia1KqSZa69PFGNHibnfO1ymlIoBOQBtt\nu/tjjwM1bnjbFzhpUJZio5Ry5mpRn6u1/snoPMWgBdBFKfUE4AaUV0rN0Vr3NziXRck+9nuklEoE\nTFrr0thIqMCUUh2AL4BWWusko/NYi1LKiasXh9sAJ4BooK/WOt7QYFakrs5QZgLntdavGJ2nuF2b\nsb+mte5kdBZLkzV2cTcTgXLAb0qpOKXUFKMDWcO1C8QvAqu4ehHxB1su6te0AAYAj1773MZdm8mK\nUk5m7EIIYWNkxi6EEDZGCrsQQtgYKexCCGFjpLALIYSNkcIuhBA2Rgq7EELYGCnsQghhY6SwCyGE\njfk/IRSLGlab0c4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x110dd5400>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = gluon.loss.EpsilonInsensitive(epsilon=0.5)\n",
    "with ag.record():    # start recording\n",
    "    theloss = loss(output, thelabel)\n",
    "theloss.backward()   # and compute the gradient\n",
    "\n",
    "plt.plot(output.asnumpy(), theloss.asnumpy(), label='Epsilon-insensitive loss 0.5')\n",
    "plt.plot(output.asnumpy(), output.grad.asnumpy(), label='Gradient of Epsilon-insensitive loss 0.5')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## LogCosh Loss\n",
    "\n",
    "An obscure variant among loss functions is the LogCosh loss. The key idea is to smooth out the L1 loss such that the loss becomes continuously differentiable even at $0$. This is accomplished by computing the softmax between $y-f$ and $f-y$, i.e. to compute $\\log \\cosh (y-f)$. The results are exactly as expected. Note that to compute it, we use a numerically stable variant $\\log \\cosh x = |x| + \\log (1+ \\exp(-2x))/2$. This ensures that large values of $x$ do not lead to divergent terms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8jef/x/HXlR0ZCBFqxyaJkMSooq2iZmlrE6pWS1V3\n+/X7+nZPbdFBUWqPWh10KS1aRRKxY6/ESIIsSci4fn+cSGlFTpKT3Dknn+fjcR4n55x7vE/wcZ37\nvs7nVlprhBBC2A47owMIIYSwLCnsQghhY6SwCyGEjZHCLoQQNkYKuxBC2Bgp7EIIYWOksAshhI2R\nwi6EEDZGCrsQQtgYByN2WrlyZV2nTh0jdi2EEFYrPDw8Xmvtnd9yhhT2OnXqEBYWZsSuhRDCaiml\nTpuznByKEUIIGyOFXQghbIwUdiGEsDFS2IUQwsZIYRdCCBsjhV0IIWyMFHYhhLAxVlXYd5+5wszf\njhsdQwghCmV/TGKJ7MeqCvu63TG892MU30TGGB1FCCEK5JvIGHp+so31e88X+76sqrBP7tGUVnW9\neHHVXvacTTA6jhBCmGVvdAIvrtpLqzpedG7qU+z7s6rC7uRgx8whLans7syYRWHEJqUbHUkIIe4o\nNjmdMQvDqezuzMyhLXFyKP6ya1WFHaCSuzNzQoNJSstkzKJw0jOyjI4khBC3dS0zi3GLwklMy2BO\naDCV3J1LZL9WV9gBmt7lyUf9mxN5NoHJa/ejtTY6khBC3EJrzf+t3U/EmQQ+6t+cpnd5lti+rbKw\nA3Tzr8akBxqwOiKaL7edNDqOEELcYv4fp/g6PJqJnRrQzb9aie7bags7wMT7G9DNrypvbzjEb4dj\njY4jhBAAbD0ax5vrD9K1mQ+TOjUo8f1bdWG3s1N82L85jap68tSy3RyPSzE6khCijDsZf5UJS3fT\n0MeDj/oHYmenSjyDxQq7UspeKbVbKfW9pbZpjnJODswJDcLR3o7RC8JITMsoyd0LIUSu5PQMRi8M\nw07BnNBg3JwNuZaRRUfsTwOHLLg9s9WoWI6ZQ1py5nIqE5ftJitbTqYKIUpWVrbm6eWRnIq/yudD\ngqjpVc6wLBYp7EqpGkAPYK4ltlcYrX0r8fpDfvx+JI73fowyKoYQooya+vNhNkXF8r9eTWlbr5Kh\nWSz1OWEa8CLgYaHtFcrg1rWIupDE7C0naOTjwSNBNYyMI4QoI76JjGHmb8cZ3LoWQ9vUNjpO0Ufs\nSqmeQKzWOjyf5cYopcKUUmFxcXFF3W2e/tuzKW19K/HKmn1EnLlSbPsRQgi4tV3Aq72aoVTJnyz9\nJ0scimkH9FZKnQKWA/crpRb/cyGt9WytdbDWOtjb29sCu709R3s7Ph/SEp/yzoxdFM6FRGk7IIQo\nHka0CzBHkVNorV/RWtfQWtcBBgKbtNZDi5ysCCq6OTE3NITUa5mMWRQmbQeEEBZnVLsAc5SO/16K\nQaOqHnw8IJC90Ym8vHqvtB0QQliMke0CzGHRwq61/k1r3dOS2yyKLs2q8nyXhqyLPMcXW04YHUcI\nYSOMbBdgDpsdsd8w/r769Ayoxns/RrEp6qLRcYQQVs7odgHmsPnCrpTig0eb07SaJxOXRXIsNtno\nSEIIK1Ua2gWYw+YLO4Crkz1zQoNxcbRj1IIwElKvGx1JCGFlSku7AHOUicIOcFcFV2YNDSImIY0J\nS3eTmZVtdCQhhJUoTe0CzFFmCjtAcB0v3urjz7Zj8by9QdoOCCHMU5raBZij9H6WKCb9Q2py6EIS\n8/44SeOqHvQPqWl0JCFEKVba2gWYo0yN2G+Y3L0J99SvzOR1+wg7ddnoOEKIUqo0tgswR5ks7A72\ndnw6uAXVK7gybnE4MQlpRkcSQpQypbVdgDmsJ6mFVSjnxNzhwaRnZDNmYRhp16XtgBDCpDS3CzBH\nmS3sAPWreDBjUCAHzyfxwqo90nZACIHWmsmluF2AOcp0YQe4v7EPL3ZtzPd7z/P5b8eNjiOEMNi8\nP06xqhS3CzBHmZsVczvjOvoSdSGJD346TIMq7nRpVtXoSEIIA2w9GsdbpbxdgDnK/IgdTG0H3nsk\ngIAa5XlmRSSHL0jbASHKGmtpF2AOKew5XBztmT0smHLODoxauIsrV6XtgBBlhTW1CzCHFPabVC3v\nwuxhQVxMusaTSyLIkLYDQtg8a2sXYA4p7P/QolZF3unrz/YTl3jz+4NGxxFCFDNraxdgDuv+vFFM\nHgmqQdSFJOZsPUmjqp4Mbl3L6EhCiGJgje0CzCEj9jy83K0JHRt6M+Wb/ew4ccnoOEIIC7PWdgHm\nkMKeB3s7xYxBLajlVY4nlkQQfSXV6EhCCAux5nYB5rCtd2Nh5V0dmTM8mIysbEYtCOPqtUyjIwkh\nisja2wWYQwp7Pup5u/PJoBYcuZjM81/vITtb2g4IYa1soV2AOaSwm+HeRlV4pVsTfth/gU82HTM6\njhCikGyhXYA5ZFaMmUa1r8uhC0l8vPEIjaq686Cf7f6lEMIWbTliG+0CzCEjdjMppXi7rz+BNSvw\nzIo9HDyXZHQkIYSZTO0CImyiXYA5pLAXgKntQBCerg6MXhjGpZRrRkcSQuQjKT2DUQt24WBvZxPt\nAswhhb2Aqni6MHtYMPEp13hiSQTXM6XtgBClVVa25ulluzl9KZXPh7S0iXYB5pDCXgjNa1bg/UcD\n2HnyMq9+d8DoOEKIPLz/UxSbD8fx2kPNaONrG+0CzGH7n0mKyUOB1Tl0PplZvx+nSTVPhtnQ15GF\nsAVrd0fzxe8nGNamNkNal61/n0UesSulXJRSO5VSe5RSB5RSr1kimDV4oWsj7m9chde+PcCfx+ON\njiOEyBF5NoGXVu+jja8XU3o1NTpOibPEoZhrwP1a6+ZAIPCgUqqNBbZb6tnbKaYPDKROZTfGL4ng\nzCVpOyCE0S4mpTNmYRg+ns58PiQIR/uyd8S5yO9Ym6TkPHTMuZWZr2d6uDgyNzSYbA2jF4aRIm0H\nhDBMekYWYxaa2n/MDQ3By83J6EiGsMh/ZUope6VUJBAL/KK13mGJ7VqLOpXd+GxwS47FpfDMikhp\nOyCEAbTWvLJmH3uiE/l4QCCNqnoYHckwFinsWussrXUgUANopZTy++cySqkxSqkwpVRYXFycJXZb\nqtzToDKTuzfhl4MXmbbxiNFxhChzZm85wdrdMTzfpWGZvyC9RQ8+aa0TgN+AB2/z2mytdbDWOtjb\n29uSuy01HmtXh/7BNZix6Rjf7z1ndBwhyozNUbG8+2MUPQOqMf6++kbHMZwlZsV4K6Uq5PzsCjwA\nRBV1u9ZIKcUbffwIql2R57/ew/6YRKMjCWHzjsUmM3HZbppW8+SDR5vb1AUzCssSI/ZqwGal1F5g\nF6Zj7N9bYLtWydnBnllDg6hYzokxC8OIS5a2A0IUl8TUDEYtCMPZ0dQuwNXJ3uhIpYIlZsXs1Vq3\n0FoHaK39tNavWyKYNfP2cGZOaDCXU68zbnE41zKzjI4khM3JzMpmwrIIYhLS+GJYEHdVcDU6UqlR\n9iZ4lhC/6uWZ2q854aevMGXdAbSWmTJCWNLbG6LYejSet/r4E1Tby+g4pYq0FChGPQPuIup8Mp9u\nPkaTah6MaFfX6EhC2ISVu84y74+TjGxXl/4hNY2OU+rIiL2YPdu5IZ2b+vDG+kNsOyptB4QoqrBT\nl5m8bh/tG1TmP90bGx2nVJLCXszs7BQfDwiknrcb45dGcCr+qtGRhLBaMQlpjFscTvUKrnw6qCUO\nZbBdgDnkt1IC3J0dmBsaglIwamEYyekZRkcSwuqkXTe1C0jPyGbu8GDKl3M0OlKpJYW9hNSqVI7P\nh7TkZPxVnl4eSZa0HRDCbFprnl+1h4Pnk5gxKJD6VcpuuwBzSGEvQXfXq8yrvZqyKSqWqT8fNjqO\nEFbjs83HWL/3PC892Jj7G/sYHafUk1kxJWxom9ocPJ/MzN+O07iqBw8FVjc6khCl2s8HLjD15yP0\nCbyLsR18jY5jFWTEXsKUUrzWuxmt6njx4qq97I1OMDqSEKVW1IUknlkRSfMa5Xn3kQBpF2AmKewG\ncHKwY+bQllR2d2bMwnBik9KNjiREqXP56nVGLwzDzdmBL4YF4+Io7QLMJYXdIJXcTW0HEtMyGLMo\nnPQMaTsgxA0ZWdk8uSSci0nX+GJYEFXLuxgdyapIYTdQ07s8+ah/cyLPJjB57X5pOyBEjte/O8hf\nJy7z7sP+tKhV0eg4VkcKu8G6+Vdj0gMNWB0RzZfbThodRwjDLdlxmkV/nWZsB18eblnD6DhWSQp7\nKTDx/gZ086vK2xsO8fsR27u6lBDm2nHiEv/75gD3NvLmxQelXUBhSWEvBezsFFP7NaehjwcTlkZw\nIi4l/5WEsDFnL6fyxJIIalUqx4xBLbC3kxkwhSWFvZRwc3ZgTmgwjvZ2jFoYRmKatB0QZcfVa5mM\nXhhGZlY2c0OD8XSRdgFFIYW9FKnpVY6ZQ1py5lIqE5ftlrYDokzIztY8uzKSIxeT+XRwS3y93Y2O\nZPWksJcyrX0r8fpDfvx+JI73fiyTl44VZcz0X4/y04GLTO7RlA4NbfNC9yVNWgqUQoNb1+LQ+SRm\nbzlBIx8PHgmSmQHCNm3Yd57pvx6lX1ANRrarY3QcmyEj9lJqSq+mtPH14pW1+9h95orRcYSwuAPn\nEnlu5R5a1qrAm339pF2ABUlhL6Uc7e34fEgQPp7OjF0UzoVEaTsgbEd8yjXGLAynQjlHZg0LwtlB\n2gVYkhT2UszLzYm5oSFcvZbJ2EVh0nZA2ITrmdk8sTicS1evMSc0mCoe0i7A0qSwl3KNqnrw8YBA\n9kQn8vLqvdJ2QFg1rTVTvtnPrlNX+ODR5vhVL290JJskhd0KdGlWlee7NGRd5Dm+2HLC6DhCFNrC\n7adZvussE+6rT6/mdxkdx2ZJYbcS4++rT4+Aarz3YxSbo2KNjiNEgf1xLJ7Xvz9I56Y+PNu5odFx\nbJoUdiuhlOKDRwNoWs2Tict2cyw22ehIQpjt9KWrPLkkgnrebnw8IBA7aRdQrKSwW5FyTg7MDg3G\n2dGOUQvCSEyVtgOi9EtOz2DUgjCUgrmhIbg7y9dnipsUditTvYIrs4YGEZOQxoRlEWRmZRsdSYg8\nZWVrnl4eycn4q3w+pCW1KpUzOlKZIIXdCgXX8eKtPv5sPRrP2xuk7YAovd7/MYpNUbG82rsZd9er\nbHScMqPIhV0pVVMptVkpdUgpdUAp9bQlgok76x9Sk8fa1WHeHydZGXbW6DhC/Muq8Gi+2HKC0La1\nGdqmttFxyhRLjNgzgee01k2ANsB4pVRTC2xX5GNy9ybcU78y/7d2P+GnLxsdR4hc4acv8581+2hX\nvxL/7SnloKQVubBrrc9rrSNyfk4GDgHVi7pdkT8Hezs+HdyCahVcGLsognMJaUZHEoLoK6mMXRTO\nXRVc+GxwSxzt5YhvSbPob1wpVQdoAey4zWtjlFJhSqmwuDi5/JulVCjnxNzQYNIzshizKIy069J2\nQBjHdMGMcK5lZjN3eAgVyjkZHalMslhhV0q5A6uBSVrrpH++rrWerbUO1loHe3tLz2VLauDjwfSB\ngRw4l8Tzq/ZI2wFhiBsXzDh8IYlPBrWgfhW5YIZRLFLYlVKOmIr6Eq31GktsUxRMpyY+vPRgY9bv\nNfW3FqKkfbzxSO4FM+5tVMXoOGVakb8poExNlL8EDmmtPyp6JFFYYzv4cuRiMtM2HqVBFQ96BFQz\nOpIoI76JjOGTTccYGFJTLphRClhixN4OGAbcr5SKzLl1t8B2RQEppXjnYX+Ca1fkua8j2RudYHQk\nUQbsOZvAi6v20qqOF68/JBfMKA0sMStmm9Zaaa0DtNaBObcNlggnCs7ZwZ5Zw4Ko5ObM6IVhcoEO\nUawuJKYzemEY3h7OzBzaEicHmQFTGsifgg2q7O7M3OHBJKdnykwZUWxuzMS6ei2TucODqeTubHQk\nkUMKu41qUs2T6QNbsC8mkRdkpoywMK01L6zay76YRKYNbEHjqp5GRxI3kcJuwzo3Nc2U+X7veWb8\neszoOMKGfLrpGN/tOceLXRvTuamP0XHEP0j/TBt3Y6bMxxuPUL+Ku8yUEUX24/7zfPjLEfq2qM64\njr5GxxG3ISN2G3djpkxQzkyZfdGJRkcSVuzAuUSeWbGHwJoVeOdhf5kBU0pJYS8DnB3s+SJnpsyo\nhbu4mCQzZUTBxSVfY/SCMCqUc2R2aBAujvZGRxJ5kMJeRtw8U2b0QpkpIwrmWmYWYxeFcTn1OnNC\ng6ni4WJ0JHEHUtjLEJkpIwpDa80ra/YRcSaBD/sF4le9vNGRRD6ksJcxnZv68GJXmSkjzDdn6wnW\nRMQw6YEGcvLdSsismDJoXEdfjsbKTBmRv40HL/LOD1H08K/GxPsbGB1HmElG7GWQzJQR5jh4LomJ\ny3fjd1d5pvZrjp2dzICxFlLYyyiZKSPuJDYpnVELduHp4sjc4cG4OskMGGsihb0Mu6WnzMIw0jNk\npoww9YAZvTCMK6kZzB0ejI+nzICxNlLYy7gbM2X2xiTywqq9MlOmjMvO1jy3cg97YxKZPlBmwFgr\nKewid6bMd3vO8ckmmSlTlk3beIT1+87zSrfGdGlW1eg4opBkVowA/p4p89EvR6jnLTNlyqJ1u2OY\nsekYA4JrMrq99ICxZjJiF8CtM2WeXRnJ7jNXjI4kSlDYqcu8uGovbXy9eKOPXAXJ2klhF7mcHeyZ\nPSwIH08XRi8M4+zlVKMjiRJw9nIqYxeFU72iK7OGBslVkGyA/AmKW1Ryd2beiBCuZ2Yz8qtdJKVn\nGB1JFKOk9AxGfrWLzGzNl8ODqVDOyehIwgKksIt/qV/FnVlDgzgZf5XxSyLIyMo2OpIoBplZ2UxY\nupuT8VeZOaQlvt7uRkcSFiKFXdzW3fUr8/bD/mw9Gs+Ubw7INEgb9Mb3B9lyJI43+/hxd/3KRscR\nFiSzYkSe+gfX5FT8VT7/7Ti+ld0Y3UFmStiKBX+eYsH204xuX5eBrWoZHUdYmBR2cUfPd2nE6Uup\nvP3DIWpVKkdXmdts9X47HMtr3x3ggSY+vNytidFxRDGQQzHijuzsFB/2b07zGhV4evlu9kYnGB1J\nFMGBc4mMXxJB46qeTB8YiL009rJJUthFvlwc7ZkTGkxld2ceXxBGTEKa0ZFEIZxPTGPkV7vwdHVk\n/mMhuDnLB3ZbJYVdmMXbw5n5I0JIv57F41/tIlmmQVqV5PQMHpu/i6vXspj/WIg09rJxUtiF2Rr4\nePD50JYcjU1hwtLdZMo0SKuQkZXN+KW7ORabwsyhLWlc1dPoSKKYSWEXBdK+gTdv9vHj9yNxvPbd\nQZkGWcpprfnvuv1sORLHW339aN/A2+hIogRYpLArpeYppWKVUvstsT1Rug1qVYuxHXxZ9Ndp5m49\naXQccQef/3ac5bvOMuG++gwIkWmNZYWlRuxfAQ9aaFvCCrz0YGN6BFTjrQ2H+HbPOaPjiNv4JjKG\nD346zEOBd/Fcl4ZGxxElyCKnxbXWW5RSdSyxLWEd7OwUH/ZrTlzyNZ5fuQdvd2fa1qtkdCyRY+fJ\ny7zw9V5a1fXi/UcDjOnWmJ0NWdchOxN0FmRngc7Ouc+69f52z+nsnJsG9N/38O/ncg8J5vUcBdvG\nneR7+DGf12u2AQ+ffLZRNCU230kpNQYYA1CrlnwktAUujvbMGRbMo7P+ZMyiML4e11ZOzJUCx+NS\nGLMojBpersweFoSzQz7XK9UariVDajykXobUS5CWANeT4VoKXE/Jub/pcUY6ZF2DzHTIvJ7z83XT\n46zrkHkNsmXm1G0NWV3shV1Z6uRXzoj9e621X37LBgcH67CwMIvsVxgvJiGNhz//A4Vi7fi7qVbe\n1ehIZVZsUjoPz/yTtOtZrH2yHbW8XCH5Alw5CYkxkHgWEqMhKcb0ODXeVMizrt9hqwqcPcDJHZzd\nTfeO5cDBCeydb7rPud3ynBPYOYCyBzv7nHu7fzx2yPnZ7qbncu6VHaicDErddH+753KeN+c5hRnb\nuJN8Xr/T+hVqmX6fhaCUCtdaB+e7nBR2YQkHzyXR/4vtVK/gyspxbSnv6mh0pDInKSGO9+YuxSsp\niscaZeCVehLij8K1xFsXdK0InjWgfHVw84ZylcCtsum+XM69a4W/C7ljOTMKnSgJUthFifvjWDwj\n5u+kZa2KLHy8Vf6HAEThaQ2XjsOJzXB2B9kxEdhdPv736+5VoXID8G4ElRtBJV8oXxM8q5uKtbBK\n5hZ2ixxjV0otA+4FKiulooH/aa2/tMS2hfVoV78yHzzanEkrInlu5R5mDGyBnfQisZzrqXDsFzj6\nMxz/DZKiAdAe1dibXY+fM4K5u2MX7mnfyTQqF2WWpWbFDLLEdoT169OiOucT03nvxyiqlXdhco+m\nRkeybhnpcOQHOLDOVNAzUsGlPNTtCO2fRfvey2vb0vhq+2n+070x93SoZ3RiUQpIFyBhceM6+nIh\nMY05W09S2d2ZsR2l2BRY3BEInw97lkHaFdOx8OaDoOlDULsd2Jv+6c787RhfbT/NqHvqMkaKusgh\nhV1YnFKKKb2aEX/1Ou/8EEV5V0e5mIM5tIaTv8PWj0z3do7QpCe0HA51O5hmitzk67CzvP+j6QtI\n/+kufdXF36Swi2Jhb6f4uH8gKemZ/GftPjxdHenuX83oWKWT1qbDLL+/BzHhphOfnf4HLYaB++17\nu/x66CIvr9lH+wam8xpyLkPcTAq7KDZODnbMHNqSYV/u5Onlu/FwcZAmVP90bjf8/F84tRUq1Iae\nH0PzweCYd1vdP4/F88SSCPzu8mTm0CCcHKSXn7iV/I0QxaqckwPzhodQz9udsYvCiThzxehIpcPV\nS7D2CZh9L8QehO5T4alwCB55x6IeceYKoxaGUbeSG1891gp3uViGuA0p7KLYlS/nyMLHW+Ht4cxj\n83dx+EKy0ZGMozXsXQmfhcC+r6HdJJi4G1qNBvs7f6nr0PkkRszbibeHM4seb0VFN6cSCi2sjRR2\nUSKqeLiw+PHWuDjaMezLHZy+dNXoSCUvJRaW9oc1o8HLF8Zugc6vmaYv5uNEXArDvtyBm7MDix9v\nTRW5ApK4AynsosTU9CrHosdbk5GVzaDZf3H2cqrRkUrO8U0wsx2c3AIPvgcjfwIf8+b4R19JZejc\nHWgNix5vTU2vcsUcVlg7KeyiRDX08WDxqNZcvZ7FwNl/EX3Fxot7dhZsfA0W9YVyXjB6M7QZ96+p\ni3k5l5DGkLk7SL6WycLHW1G/irQDEPmTwi5KXLO7yrNkVGuS0zMYNOcvziWkGR2peKQnwtIBsO0j\n01z00ZvNHqWDqagPmvMXl1Ous2BkK5rdlf8hGyFACrswiF/18ix6vDUJV03F/UJiutGRLOvScZjT\nydSkq+c06D0DnMw/hHJLUX+8FS1rSe8XYT4p7MIwzWtWYMHjrbiUcp1Bc/7iYpKNFPezu2DO/ZB2\nGUK/heDHCrT6zUV9oRR1UQhS2IWhWtaqyIKRIcQmpdP/i+3Wf0L16EZY2Nt0PH3Ur1CnXYFW/2dR\nbyFFXRSCFHZhuKDaXiwe1ZorV6/T/4vtHI9LMTpS4exbBcsGQKV6plkvXnULtPrJ+Kv0m7Vdiroo\nMinsolRoUasiK8a2JSMrmwFfbOfguSSjIxVMxCJYPQpqtoYR68G9SoFWP3guiX6ztpOWkcXS0W2k\nqIsikcIuSo0m1TxZMbYtjvZ2DJy93XraD0QuhW+fgnr3w9DVZn3h6GZhpy4zYPZ2HO0VK8e2xb+G\nzH4RRSOFXZQq9bzdWTm2LRXdnBg6dwebo2KNjnRne1fCuifBtyMMXAKOBbuQ92+HYxn65Q4quzvz\n9bi2Mk9dWIQUdlHq1PQqx9dj2+Lr7cbjC3ax+K/TRke6vf1rYO1YqHMPDFxW4KK+fOcZRi0Iw7ey\nO1+Pa0uNivKNUmEZUthFqVTF04UVY9rSsaE3/7duP+/+EEV2tmUuvG4RxzfBmjGmY+qDVxRojnp2\ntubdH6J4ec0+2tarxIqxbajs7lyMYUVZI4VdlFpuzg7MCQ1mcOtazPr9OE8t303a9SyjY5l6qK8Y\nBpUbwqDl4ORm9qrpGVlMWBbBrN+PM7h1LeaPCMHD5c5dHYUoKGnmLEo1B3s73urjRy2vcrz3YxTH\nY1OYPSyYWpUMOmxx6Tgs6QeuFU0nSl0rmL3q2cupPLEknAPnkpjcvQmj2tdFKbnykbA8GbGLUk8p\nxbiO9Zg3IoRzCWn0+nQbvx024KRq8kVY/LCpsdfQNeBp/qX+fjscS69Pt3H6UipzQ4MZ3cFXiroo\nNlLYhdW4r1EVvnvqHqqVd+Gxr3bx8S9HyMzKLpmdZ6TBsoGm4j54JXg3NGu1zKxsPv7lCI99tYuq\nni58N+EeOjXxKeawoqyTwi6sSu1Kbqx9sh19W1Rn+q9HeXTWdk7FF/NFO7SGb8bDuQh4ZC7UDDFr\ntdOXrtL/i+1M//UofVtUZ+2T7ahT2fzj8UIUlhR2YXVcnez5qH8gnwxqwYm4FLrP2MqynWfQuphm\nzWz5APavhk5ToEnPfBfXWrN85xm6Td/K0dgUpg8M5KP+gbg6mdeDXYiikpOnwmr1an4XwXUq8tzK\nPbyyZh9rIqJ5s48/jap6WG4nB7+BzW9BwAC459l8Fz98IZn/W7ePXaeucHe9Skzt15y7KhRsfrsQ\nRaWKbZRzB8HBwTosLKzE9ytsU3a2ZlV4NO/8cIik9Eweu7sOE+6vT4VyRbzY87lImPcgVPWD4d+D\nY97XGU1Ivc5nm48x749TeLo48Eq3JjwaVAM7OzlBKixHKRWutQ7Odzkp7MJWXLl6nXd/iGJl+Fnc\nnRwY1d6Xx9vXxd25EB9Mky/A7PtA2cHoTeBx+xOeKdcymbftJHO2nCDleib9g2rycrfGVHQr4n8q\nQtxGiRZ2pdSDwHTAHpirtX73TstLYRfFKepCEh/9fISfD17Ew8WBfkE1CW1b2/wTlxlp8FUPiD1k\nar9bLeBfi5yKv8rC7af5OvwsyemZdGnqw7NdGtK4qqeF340Qfyuxwq6UsgeOAJ2BaGAXMEhrfTCv\ndaSwi5Jzf3omAAAVC0lEQVSwNzqBL7edZP3e82RpTas6XvQMqEaXZlXx8czjsIrWpva7+1fBgMXQ\npFfuS7FJ6fx04ALr951nx8nL2CtFd/9qjGpfl4Aa5n9RSYjCKsnC3hZ4VWvdNefxKwBa63fyWkcK\nuyhJsUnpLN91lu/2nONorOkiHnUruxFSpyKNqnpSt3I5qnq64u7sgFfEdNz/eJfYkBc5UG80J+Kv\ncvhCErtOXeFkzrTK+lXc6RVwFwNb1cz7PwghikFJFvZHgQe11qNyHg8DWmutJ+S1jhR2YZQjF5P5\n/XAcO05eJuz0ZRJSM3Jf62q3ky+cprE2qx3PZDwJmE58VijnSHDtirSq68W9jarQ0MeCs26EKABz\nC7slpjve7rT/v/63UEqNAcYA1KpVywK7FaLgGvp40NDHg9EdfNFac/nqdU5dukr6md20/u0L4tz9\nyb57BjPLuVHF05k6ldzwcnOSr/8Lq2KJwh4N1LzpcQ3g3D8X0lrPBmaDacRugf0KUSRKKSq5O1NJ\nJ8DqieDmhfeo1TySxwwYIayFJb55ugtooJSqq5RyAgYC31pgu0IUv4x0WD4Y0i7DoGV5TmsUwpoU\necSutc5USk0AfsI03XGe1vpAkZMJUdy0hm8nQEwY9F8E1ZobnUgIi7BISwGt9QZggyW2JUSJ2ToV\n9n0N9/8XmvY2Oo0QFiNNwETZdPBb2PQm+PeH9s8ZnUYIi5LCLsqec5Gmi1DXCIHen4DMeBE2Rgq7\nKFsSo2HZIHD1ggFL7tjYSwhrJW17RdmRlgCLH4VryTDyB5kBI2yWFHZRNmReg+VD4NIxGLoKqvrf\ncfGMjAyio6NJT08voYBC/M3FxYUaNWrg6OhYqPWlsAvbl50Na8fB6W3w8BzwvTffVaKjo/Hw8KBO\nnTryrVNRorTWXLp0iejoaOrWrVuobcgxdmHbtIZf/gsH1sADr0JAf7NWS09Pp1KlSlLURYlTSlGp\nUqUifVqUwi5s25YPYPun0GoMtJtUoFWlqAujFPXvnhR2Ybv+mGG6XmnzwfDge1Y3rdHd3d0i2zly\n5Ajdu3enfv36NGnShP79+3Px4sUCbePUqVP4+fnlu9yrr77K1KlTCxtVWIgcYxe2aecc0yGYZg/D\nQ5+CXdkcw6Snp9OjRw8++ugjevUyXTRk8+bNxMXF4eMjs4JsVdn82y5s2665sOF5aNQDHp4NdvZG\nJ7KY06dP06lTJwICAujUqRNnzpwB4Pjx47Rp04aQkBCmTJmSO9pfunQpbdu2zS3qAPfddx9+fn6k\np6fz2GOP4e/vT4sWLdi8eTMABw4coFWrVgQGBhIQEMDRo0cByMrKYvTo0TRr1owuXbqQlpZ2x6yR\nkZG0adOGgIAA+vbty5UrVwCYMWMGTZs2JSAggIEDBwLw+++/ExgYSGBgIC1atCA5Odmyv7gyRkbs\nwrb8MR1+mQINu0G/+WBfuOliN3vtuwMcPJdkgXB/a3qXJ//r1azA602YMIHQ0FCGDx/OvHnzmDhx\nIuvWrePpp5/m6aefZtCgQcyaNSt3+f379xMUFHTbbX322WcA7Nu3j6ioKLp06cKRI0eYNWsWTz/9\nNEOGDOH69etkZWVx8eJFjh49yrJly5gzZw79+/dn9erVDB06NM+soaGhfPLJJ3Ts2JEpU6bw2muv\nMW3aNN59911OnjyJs7MzCQkJAEydOpXPPvuMdu3akZKSgouLfHGsKGTELmyD1rD5bVNRb/YwDFgE\nDs5Gp7K47du3M3jwYACGDRvGtm3bcp/v168fQO7r+dm2bRvDhg0DoHHjxtSuXZsjR47Qtm1b3n77\nbd577z1Onz6Nq6srAHXr1iUwMBCAoKAgTp06lee2ExMTSUhIoGPHjgAMHz6cLVu2ABAQEMCQIUNY\nvHgxDg6msWW7du149tlnmTFjBgkJCbnPi8KR356wflmZ8MMLEDYPWgyFXjMsevilMCPrkpLf7Ilm\nzZrx+++/3/a1vC6LOXjwYFq3bs369evp2rUrc+fOxdfXF2fnv/+jtLe3z/dQTF7Wr1/Pli1b+Pbb\nb3njjTc4cOAAL7/8Mj169GDDhg20adOGjRs30rhx40JtX8iIXVi79CRYNsBU1NtNgl6f2NQx9X+6\n++67Wb58OQBLlizhnnvuAaBNmzasXr0aIPd1MBXpP//8k/Xr1+c+9+OPP7Jv3z46dOjAkiVLANPM\nmTNnztCoUSNOnDiBr68vEydOpHfv3uzdu7fAOcuXL0/FihXZunUrAIsWLaJjx45kZ2dz9uxZ7rvv\nPt5//30SEhJISUnh+PHj+Pv789JLLxEcHExUVFThfkECkBG7sGYJZ0wNvWIPQa/pEDTC6EQWlZqa\nSo0aNXIf3zhUMXLkSD744AO8vb2ZP38+ANOmTWPo0KF8+OGH9OjRg/LlywPg6urK999/z6RJk5g0\naRKOjo4EBAQwffp0nnzyScaNG4e/vz8ODg589dVXODs7s2LFChYvXoyjoyNVq1ZlypQpJCUV/BzD\nggULGDduHKmpqfj6+jJ//nyysrIYOnQoiYmJaK155plnqFChAv/973/ZvHkz9vb2NG3alG7dulnm\nl1hGqbw+jhWn4OBgHRYWVuL7FTbkyM+wZjTobOi/AOrdb9HNHzp0iCZNmlh0m8UpNTUVV1dXlFIs\nX76cZcuW8c033xgdSxTB7f4OKqXCtdbB+a0rI3ZhXbKzTCdJt04FH39TUa9Uz+hUhgsPD2fChAlo\nralQoQLz5s0zOpIwkBR2YT3ij8G6cRC9C1oMg+4fgKOr0alKhfbt27Nnzx6jY4hSQgq7KP2ys2Hn\nF7DxVXBwgYfnQkA/o1MJUWpJYRelW3QYbHgBzkVAwwdNJ0k9qhqdSohSTQq7KJ2SzsOvr8OepeBe\n1dRH3b+f1TXyEsIIUthF6ZJ8EbZ9bJqXrrNNc9M7PA/OHkYnE8JqyBeUROkQdwTWPw/Tm8PO2abR\n+YRd0Pm1MlvUL168yODBg/H19SUoKIi2bduydu3aIm3z5ra6U6ZMYePGjYXaTmRkJBs2bCjweoMG\nDSIgIICPP/44z1xFNXXqVBo3boyfnx/Nmzdn4cKFBd7GiBEjWLVqVb7LWaq1sqXJiF0YJysDjv5i\n6sZ4/FewdzIV9PbPlfkpjFpr+vTpw/Dhw1m6dClg6uz47bff/mvZzMzMQvVWef311wudLzIykrCw\nMLp37272OhcuXODPP//k9OnThd5vfmbNmsUvv/zCzp078fT0JDExkXXr1hXb/korGbGLkpWdDWd2\nwPrnYGpDWD4ILh6A+/4PnjkIfT4v80UdYNOmTTg5OTFu3Ljc52rXrs1TTz0FwFdffUW/fv3o1asX\nXbp0ISUlhU6dOtGyZUv8/f1v+XLSW2+9RaNGjXjggQc4fPhw7vM3j0rDw8Pp2LEjQUFBdO3alfPn\nzwNw77338tJLL9GqVSsaNmzI1q1buX79OlOmTGHFihUEBgayYsWKW7Ln1Q64S5cuxMbGEhgYmNtq\nID8fffQRfn5++Pn5MW3atNzn33jjDRo3bkznzp0ZNGhQ7mj/7bff5vPPP8fT0xMwtTYYPnw4AL/+\n+istWrTA39+fkSNHcu3aNQBefvnl3DbCzz//fO4+tmzZwt13342vr2++o3etNS+88AJ+fn74+/vn\n/k7Onz9Phw4dCAwMxM/Pj61bt5KVlcWIESNyl/3npxdLkBG7KH7piXDiNzjyExz9Ga7GmaYtNuoO\nAQOgfieLtNctNj+8DBf2WXabVf2h27t5vnzgwAFatmx5x01s376dvXv34uXlRWZmJmvXrsXT05P4\n+HjatGlD7969iYiIYPny5ezevZvMzExatmz5rza+GRkZPPXUU3zzzTd4e3uzYsUKJk+enPslp8zM\nTHbu3MmGDRt47bXX2LhxI6+//jphYWF8+umn/8qVVzvgb7/9lp49exIZGWnWryg8PJz58+ezY8cO\ntNa0bt2ajh07kpWVxerVq//1npKTk0lOTqZevX8PDNLT0xkxYgS//vorDRs2JDQ0lJkzZxIaGsra\ntWuJiopCKZXbRhhMRXnbtm1ERUXRu3dvHn300TyzrlmzhsjISPbs2UN8fDwhISF06NCBpUuX0rVr\nVyZPnkxWVhapqalERkYSExPD/v37AW7Zp6VIYReWpTUkxcDZnXDmLzizHS7uN50IdSkP9R+ABl2h\nUTdw8TQ6rdUYP34827Ztw8nJiV27dgHQuXNnvLy8ANOI8T//+Q9btmzBzs6OmJgYLl68yNatW+nb\nty/lypUDoHfv3v/a9uHDh9m/fz+dO3cGTBfUqFatWu7rDz/8MJB/q94btm3blvvJ4uZ2wDdG0eba\ntm0bffv2xc3NLTfH1q1byc7O5qGHHsptJ3zjIiJa6zy7XR4+fJi6devSsGFDwNRG+LPPPmPChAm4\nuLgwatQoevToQc+ePXPX6dOnD3Z2djRt2jTfSwlu27aNQYMGYW9vj4+PDx07dmTXrl2EhIQwcuRI\nMjIy6NOnD4GBgfj6+nLixAmeeuopevToQZcuXQr0ezFHkQq7Uqof8CrQBGiltZYGMGWF1pASC1dO\nwqVjcGG/qYBf2AfpOSMQx3JQIwQ6vAi+HaFGK7C3wrHEHUbWxaVZs2a53RrBNAqOj48nOPjvNiE3\nCh6YOj3GxcURHh6Oo6MjderUyb3KfX6tfbXWNGvWjO3bt9/29Rvteu3t7cnMzMw3u6X6T+W1nbye\n9/T0xM3NLbc7pTnrODg4sHPnTn799VeWL1/Op59+yqZNmwBuaVOc33vK6/UOHTqwZcsW1q9fz7Bh\nw3jhhRcIDQ1lz549/PTTT3z22WesXLnS4i0ginqMfT/wMLDFAllEaZGVaZp2eH6P6eTm7sWwZSps\neBGWDYbP74a3q8OHDWFeV/hmPEQsgIw0aNYHuk+FUZvg5TMw/Fu47xWofbd1FnWD3H///aSnpzNz\n5szc51JTU/NcPjExkSpVquDo6MjmzZtzT1B26NCBtWvXkpaWRnJyMt99992/1m3UqBFxcXG5hT0j\nI4MDBw7cMZ+Hh0eel6/Lqx1wQXXo0IF169aRmprK1atXWbt2Le3bt+eee+7hu+++Iz09nZSUlFta\nEr/yyiuMHz8+txtlUlISs2fPpnHjxpw6dYpjx44Bf7cRTklJITExke7duzNt2jSzDxPdLuuKFSvI\nysoiLi6OLVu20KpVK06fPk2VKlUYPXo0jz/+OBEREcTHx5Odnc0jjzzCG2+8QURERKH2eSdF+pem\ntT4E+Y8IRCFkZ4POguzMm25ZObebntPZtz7OvGYqsJnppltGOmSm3eY+zdTLPD3x37eMq7fP5FIe\nPKpBxbqmEXjFuuBVF7x8oWIdm+6DXtKUUqxbt45nnnmG999/H29vb9zc3Hjvvfduu/yQIUPo1asX\nwcHBBAYG5l6komXLlgwYMIDAwEBq165N+/bt/7Wuk5MTq1atYuLEiSQmJpKZmcmkSZNo1izvC4zc\nd999vPvuuwQGBvLKK68wYMCA3NfyagecnzfffPOWE6TR0dGMGDGCVq1aATBq1ChatGgBmA4pNW/e\nnNq1axMcHJzbpviJJ54gJSWFkJAQHB0dcXR05LnnnsPFxYX58+fTr18/MjMzCQkJYdy4cVy+fJmH\nHnqI9PR0tNaFPpHZt29ftm/fTvPmzVFK8f7771O1alUWLFjABx98gKOjI+7u7ixcuJCYmBgee+wx\nsrOzAXjnnXcKtc87sUjbXqXUb8Dz5h6KKXTb3t/fh32rAG06FACmn+HWx3d8jUKuVxL7y/67cFP0\nP5c8Obiamme5lM+5ed70cwXTvWtFcPcxfX3fvYrp5zLUcMva2vaWNSkpKbi7u5OamkqHDh2YPXt2\nviebrU2xtu1VSm0EbtecY7LW2uyGz0qpMcAYgFq1apm72q3cfaBKkxsbBNRNP/OPx4V57Z/LWWKb\n+b3GrcvZOeTc7HNuOY+V/U3PO9z62o2flT04OOUUbpfb3zs4y9fyhdUbM2YMBw8eJD09neHDh9tc\nUS+qfAu71voBS+xIaz0bmA2mEXuhNhI03HQTQpRpN760JW5PvqAkhBA2pkiFXSnVVykVDbQF1iul\nfrJMLCGMZ8RlI4WAov/dK+qsmLVA0boSCVEKubi4cOnSJSpVqiSzvkSJ0lpz6dIlXFxcCr0NmVgs\nxG3UqFGD6Oho4uLijI4iyiAXFxdq1KhR6PWlsAtxG46OjtStW9foGEIUipw8FUIIGyOFXQghbIwU\ndiGEsDEWaSlQ4J0qFQcU32VUik9lIN7oECVM3rPtK2vvF6z3PdfWWnvnt5Ahhd1aKaXCzOnTYEvk\nPdu+svZ+wfbfsxyKEUIIGyOFXQghbIwU9oKZbXQAA8h7tn1l7f2Cjb9nOcYuhBA2RkbsQghhY6Sw\nF4JS6nmllFZKVTY6S3FTSn2glIpSSu1VSq1VSlUwOlNxUUo9qJQ6rJQ6ppR62eg8xU0pVVMptVkp\ndUgpdUAp9bTRmUqKUspeKbVbKfW90VmKgxT2AlJK1QQ6A2eMzlJCfgH8tNYBwBHgFYPzFAullD3w\nGdANaAoMUko1NTZVscsEntNaNwHaAOPLwHu+4WngkNEhiosU9oL7GHiRYr0oaemhtf5Za52Z8/Av\noPAt50q3VsAxrfUJrfV1YDnwkMGZipXW+rzWOiLn52RMha66samKn1KqBtADmGt0luIihb0AlFK9\ngRit9R6jsxhkJPCD0SGKSXXg7E2PoykDRe4GpVQdoAWww9gkJWIapsFZttFBiou07f2HO128G/gP\n0KVkExU/cy5YrpSajOmj+5KSzFaCbnc1jTLxqUwp5Q6sBiZprZOMzlOclFI9gVitdbhS6l6j8xQX\nKez/kNfFu5VS/kBdYE/OFXVqABFKqVZa6wslGNHi8rtguVJqONAT6KRtd35sNFDzpsc1gHMGZSkx\nSilHTEV9idZ6jdF5SkA7oLdSqjvgAngqpRZrrYcanMuiZB57ISmlTgHBWmtrbCRkNqXUg8BHQEet\ntc1eTkgp5YDp5HAnIAbYBQzWWh8wNFgxUqYRygLgstZ6ktF5SlrOiP15rXVPo7NYmhxjF/n5FPAA\nflFKRSqlZhkdqDjknCCeAPyE6STiSlsu6jnaAcOA+3P+bCNzRrLCysmIXQghbIyM2IUQwsZIYRdC\nCBsjhV0IIWyMFHYhhLAxUtiFEMLGSGEXQggbI4VdCCFsjBR2IYSwMf8PHWlSKbxuQNEAAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1110051d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = gluon.loss.LogCosh()\n",
    "with ag.record():    # start recording\n",
    "    theloss = loss(output, thelabel)\n",
    "theloss.backward()   # and compute the gradient\n",
    "\n",
    "plt.plot(output.asnumpy(), theloss.asnumpy(), label='LogCosh loss')\n",
    "plt.plot(output.asnumpy(), output.grad.asnumpy(), label='Gradient of LogCosh loss')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Poisson \n",
    "\n",
    "In some cases the regression problem does not have to deal with continuous values that could be both positive or negative, but rather with *integer counts*. For instance, the number of rain drops per square meter in a given time, the number of meteorites hitting Antarctica per day, the number of Prussian soldiers that were hit by horses per week, etc. can be useful numbers to estimate. However, it is equally clear that a real valued estimate is useless: we never have 1.3 meteorites. It's only $0, 1, 2, 3, \\ldots$ or some other number. Consequently, we need a different loss function. Fortunately the Poisson distribution fits the bill quite well. In it, we assume that \n",
    "\n",
    "$$p(y|f) = \\frac{1}{y!} \\exp(y f - \\exp(f)) \\text{ and } l(y,f) = - \\log p(y|f).$$\n",
    "\n",
    "In many cases one uses an equivalent formulation with rate parameter $\\lambda = \\exp(f)$ such that we get\n",
    "$p(y|\\lambda) = \\frac{1}{y!} \\lambda^y e^{-\\lambda}$. Note that this is entirely equivalent. The only problem with the $\\lambda$-parametrization is that $\\lambda$ must be nonnegative, whereas $f$ can assume arbitrary values. **Unlike Keras and PyTorch, Gluon uses the exponential formulation**. \n",
    "\n",
    "By design, the loss function vanishes for $y = \\exp(f)$, as can be seen in the graph below (this is one of the reasons why sometimes the $\\lambda$ parametrization is preferable). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4VVW6+PHvSicJAdIgPYQSSgIBAoQiRaqoII4dpKjj\n9Y5XHR0dRX/XUWfGsV11HB0dRwURFRRFcMBGUWoooQcIBEgP6T2knazfHydgxBBSTk3ez/Pk2efs\ns/Ze7wnhPeusvfe7ldYaIYQQHZ+DtQMQQghhGZLwhRCik5CEL4QQnYQkfCGE6CQk4QshRCchCV8I\nIToJSfhCCNFJSMIXQohOQhK+EEJ0Ek7WDqAxX19fHR4ebu0whBDCriQkJORrrf2u1M6mEn54eDj7\n9u2zdhhCCGFXlFKpLWknUzpCCNFJtDjhK6U+UErlKqWONvHao0oprZTybXiulFJvKKWSlVKHlVLD\nTRm0EEKI1mvNCH8ZMPPSlUqpEGAakNZo9TVAv4afe4G32x6iEEIIU2jxHL7WeqtSKryJl14D/gis\nbbRuDrBcG2svxyuluiulArTW2a0NsLa2loyMDKqqqlq7qRCt4ubmRnBwMM7OztYORQizaNdBW6XU\nbCBTa31IKdX4pSAgvdHzjIZ1v0r4Sql7MX4LIDQ09Fd9ZGRk0LVrV8LDw7mkDyFMRmtNQUEBGRkZ\n9O7d29rhCGEWbT5oq5RyB54Cnm7q5SbWNXmnFa31u1rrWK11rJ/fr88qqqqqwsfHR5K9MCulFD4+\nPvJNUnRo7Rnh9wF6AxdG98HAfqXUKIwj+pBGbYOBrLZ2JMleWIL8nYmOrs0jfK31Ea21v9Y6XGsd\njjHJD9danwPWAQsaztaJA0raMn8vhBCdwo8vwuktZu+mNadlfgrsAiKVUhlKqbubab4BOAMkA/8G\nfteuKK3M0dGRmJgYoqKiuPnmm6msrGy2/dixYy0UmVFKSgpRUVEW7VMIYSK15+GnFyAt3uxdtTjh\na61v11oHaK2dtdbBWuv3L3k9XGud3/BYa63v11r30VpHa63t+vLZLl26cPDgQY4ePYqLiwvvvPNO\ns+137txpociEEHavIBl0PfhFmr0rudK2la666iqSk5MBePXVV4mKiiIqKorXX3/9YhtPT08AsrOz\nmTBhwsVvB9u2bcNgMLBo0SKioqKIjo7mtddeA+DgwYPExcUxZMgQ5s6dS1FREQCTJk3i8ccfZ9So\nUfTv359t27Y1G19VVRWLFy8mOjqaYcOGsWWL8WtiYmIio0aNIiYmhiFDhnDq1CkqKiq49tprGTp0\nKFFRUaxatcrkvy8hxBXkJRmXfgPM3pVN1dK5kme/TuRYVqlJ9zko0Is/XT+4RW3r6ur45ptvmDlz\nJgkJCSxdupTdu3ejtWb06NFMnDiRYcOGXWz/ySefMGPGDJ566ikMBgOVlZUcPHiQzMxMjh41XrBc\nXFwMwIIFC/jHP/7BxIkTefrpp3n22WcvfojU1dWxZ88eNmzYwLPPPsvGjRsvG+Nbb70FwJEjRzhx\n4gTTp0/n5MmTvPPOOzz00EPMmzePmpoaDAYDGzZsIDAwkPXr1wNQUlLS+l+gEKJ98k6AcgSfPmbv\nSkb4LXD+/HliYmKIjY0lNDSUu+++m+3btzN37lw8PDzw9PTkxhtv/NXoe+TIkSxdupRnnnmGI0eO\n0LVrVyIiIjhz5gwPPPAA3377LV5eXpSUlFBcXMzEiRMBWLhwIVu3br24nxtvvBGAESNGkJKS0mys\n27dv58477wRgwIABhIWFcfLkScaMGcPzzz/Piy++SGpqKl26dCE6OpqNGzfy+OOPs23bNrp162bC\n35oQokXyToB3BDi5mr0ruxrht3QkbmoX5vAbM15E3LwJEyawdetW1q9fz5133sljjz3GggULOHTo\nEN999x1vvfUWn3322cVpnctxdTX+ITg6OlJXV9ds28vFdccddzB69GjWr1/PjBkzeO+997j66qtJ\nSEhgw4YNLFmyhOnTp/P0001dViGEMJu8JIvM34OM8NtswoQJfPXVV1RWVlJRUcGaNWu46qqrftEm\nNTUVf39/fvvb33L33Xezf/9+8vPzqa+v5ze/+Q1//vOf2b9/P926daNHjx4XvyF89NFHF0f7bYnr\n448/BuDkyZOkpaURGRnJmTNniIiI4MEHH2T27NkcPnyYrKws3N3dmT9/Po8++ij79+9v3y9FCNE6\ndTVQcNoi8/dgZyN8WzJ8+HAWLVrEqFGjALjnnnt+MX8P8OOPP/Lyyy/j7OyMp6cny5cvJzMzk8WL\nF1NfXw/A3/72NwA+/PBD7rvvPiorK4mIiGDp0qVtiut3v/sd9913H9HR0Tg5ObFs2TJcXV1ZtWoV\nK1aswNnZmV69evH000+zd+9eHnvsMRwcHHB2dubtt6XGnRAWVXgatMFiCV+1ZGrCUmJjY/WlN0A5\nfvw4AwcOtFJEorORvzdhUYlr4PNF8F/bIGBIm3ejlErQWsdeqZ1M6QghhLXkJQEKfPtZpDtJ+EII\nYS15J6BHODh3sUh3kvCFEMJa8k5abP4eJOELIYR1GOqg4JTFTskESfhCCGEdRSlgqJERvhBCdHh5\nJ4xLGeHblpycHO644w4iIiIYMWIEY8aMYc2aNe3a5zPPPMMrr7wCwNNPP91sfZzmHDx4kA0bNrR6\nu9tvv50hQ4b86irfZ555hqCgoIsF39atW9fsft555x2WL1/e6v7bY9GiRaxevdqifQphcrnHjUvf\n/hbrUi68ugKtNTfccAMLFy7kk08+AYxX0DaVCOvq6nByav2v9LnnnmtzfAcPHmTfvn3MmjWrxduc\nO3eOnTt3kpqa2uTrDz/8MI8++ijHjx/nqquuIjc3FweHpscG9913X5viFqLTyzkKPXqDq6fFupQR\n/hVs3rwZFxeXXyS2sLAwHnjgAQCWLVvGzTffzPXXX8/06dMpLy9nypQpDB8+nOjoaNauXXtxu7/+\n9a9ERkYydepUkpKSLq5vPGJNSEhg4sSJjBgxghkzZpCdbbxRWFNlkmtqanj66adZtWoVMTExvypv\nfLlSydOnTyc3N5eYmJhmyy0PHDgQJycn8vPzSU1NZcqUKQwZMoQpU6aQlpYG/PKbyhtvvMGgQYMY\nMmQIt912GwA//fQTMTExxMTEMGzYMMrKytBa89hjj10sEX0h7h9//JFJkyZx0003MWDAAObNm3fF\nmkWbNm1i2LBhREdHc9ddd1FdXQ3AE088cTGWRx99FIDPP/+cqKgohg4dyoQJE5rdrxBml3sMelq2\nPph9jfC/eQLOHTHtPntFwzUvXPblxMREhg8f3uwudu3axeHDh/H29qauro41a9bg5eVFfn4+cXFx\nzJ49m/3797Ny5UoOHDhAXV0dw4cPZ8SIEb/YT21tLQ888ABr167Fz8+PVatW8dRTT/HBBx8ATZdJ\nfu6559i3bx9vvvnmr+K6XKnkdevWcd111/2qINyldu/ejYODA35+fsyePZsFCxawcOFCPvjgAx58\n8EG++uqrX7R/4YUXOHv2LK6urhfLPr/yyiu89dZbjBs3jvLyctzc3Pjyyy85ePAghw4dIj8/n5Ej\nR15MwAcOHCAxMZHAwEDGjRvHjh07GD9+fJPxVVVVsWjRIjZt2kT//v1ZsGABb7/9NgsWLGDNmjWc\nOHECpdTFWJ577jm+++47goKCLq4TwipqzxtvfDJ4rkW7bc0tDj9QSuUqpY42WveyUuqEUuqwUmqN\nUqp7o9eWKKWSlVJJSqkZpg7cWu6//36GDh3KyJEjL66bNm0a3t7egHEK6Mknn2TIkCFMnTqVzMxM\ncnJy2LZtG3PnzsXd3R0vLy9mz579q30nJSVx9OhRpk2bRkxMDH/5y1/IyMi4+HpryiTD5UslX8lr\nr71GTEwMjz76KKtWrUIpxa5du7jjjjsAuPPOO9m+ffuvthsyZAjz5s1jxYoVF6e2xo0bxyOPPMIb\nb7xBcXExTk5ObN++ndtvvx1HR0d69uzJxIkT2bt3LwCjRo0iODgYBwcHYmJimn2fSUlJ9O7dm/79\njXOgF8pKe3l54ebmxj333MOXX36Ju7v7xVgWLVrEv//9bwwGwxV/D0KYTd4J412ubHiEvwx4E2h8\nhO4HYInWuk4p9SKwBHhcKTUIuA0YDAQCG5VS/bXW7ftf1sxI3FwGDx7MF198cfH5W2+9RX5+PrGx\nP5et8PDwuPj4448/Ji8vj4SEBJydnQkPD6eqqgoApVSzfWmtGTx4MLt27Wry9daUSb6wv7a4MIff\nnKbey/r169m6dSvr1q3jz3/+M4mJiTzxxBNce+21bNiwgbi4ODZu3NhsXBfeI1z5fV5uP05OTuzZ\ns4dNmzaxcuVK3nzzTTZv3sw777zD7t27Wb9+PTExMRw8eBAfH59m36cQZpFzzLj0t2zCb809bbcC\nhZes+15rfeF/ZDwQ3PB4DrBSa12ttT6L8Wbmo0wQr8VdffXVVFVV/aKSZHM3MS8pKcHf3x9nZ2e2\nbNly8cDohAkTWLNmDefPn6esrIyvv/76V9tGRkaSl5d3MeHX1taSmJjYbHxdu3alrKysydcuVyq5\nLcaOHcvKlSsB44fapdMs9fX1pKenM3nyZF566SWKi4spLy/n9OnTREdH8/jjjxMbG8uJEyeYMGEC\nq1atwmAwkJeXx9atWy9WHW2NAQMGkJKScvGWkxfKSpeXl1NSUsKsWbN4/fXXL05dnT59mtGjR/Pc\nc8/h6+tLenp6m34XQrRbTiI4dQHv3hbt1pRz+HcBF44aBmH8ALggo2Gd3VFK8dVXX/Hwww/z0ksv\n4efnh4eHBy+++GKT7efNm8f1119PbGwsMTExDBhgvKhi+PDh3HrrrcTExBAWFvar2vkALi4urF69\nmgcffJCSkhLq6ur4/e9/z+DBlx8FTJ48mRdeeIGYmBiWLFnCrbfeevG1y5VKbos33niDu+66i5df\nfhk/P79flW82GAzMnz+fkpIStNY8/PDDdO/enf/93/9ly5YtODo6MmjQIK655hpcXFzYtWsXQ4cO\nRSnFSy+9RK9evThx4kSrYnJzc2Pp0qXcfPPN1NXVMXLkSO677z4KCwuZM2cOVVVVaK0vnnr62GOP\ncerUKbTWTJkyhaFDh7bpdyFEu+UcBf8B4OBo0W5bVR5ZKRUO/EdrHXXJ+qeAWOBGrbVWSr0F7NJa\nr2h4/X1gg9b6i0t2iVLqXuBegNDQ0BGXnioo5WqFJcnfm7CIl/tC/xkw5y2T7M5i5ZGVUguB64B5\n+udPjwwgpFGzYCCrqe211u9qrWO11rF+fn7tDUcIIWxbeS5U5Fl8/h7amfCVUjOBx4HZWuvGE9vr\ngNuUUq5Kqd5AP2BPe/oSQogOIafhREcLn6EDrZjDV0p9CkwCfJVSGcCfMJ6V4wr80HDWRrzW+j6t\ndaJS6jPgGFAH3N+eM3S01lc8w0WI9rKlu7+JDuzCGTq2nPC11rc3sfr9Ztr/FfhrW4JqzM3NjYKC\nAnx8fCTpC7PRWlNQUICbm5u1QxEdXU4iePYED1+Ld23zV9oGBweTkZFBXl6etUMRHZybmxvBwcFX\nbihEe+QctcroHuwg4Ts7O9O7t2XPVRVCCLMw1BnvY9vbOrWcpHiaEEJYSkEyGKqtNsKXhC+EEJZy\n7rBxGWCdi/4k4QshhKVkHwInN/C13F2uGpOEL4QQlpJ9yDid42idw6eS8IUQwhLq640J30rTOSAJ\nXwghLKM4BapLJeELIUSHl33IuJSEL4QQHVz2YXBwAv9BVgtBEr4QQlhC9iHwHwhObbsnhSlIwhdC\nCHPT2uoHbEESvhBCmF9pFlTmQ0CMVcOQhC+EEOZmAwdsQRK+EEKYX/YhUA5Wq6FzgSR8IYQwt+xD\n4NsfXDysGoYkfCGEMLfsg9BriLWjaHnCV0p9oJTKVUodbbTOWyn1g1LqVMOyR8N6pZR6QymVrJQ6\nrJQabo7ghRDC5pVmQVk2BI2wdiStGuEvA2Zesu4JYJPWuh+wqeE5wDUYb1zeD7gXeLt9YQohhJ3K\nTDAu7Snha623AoWXrJ4DfNjw+EPghkbrl2ujeKC7UiqgvcEKIYTdydgHDs7QK9rakbR7Dr+n1job\noGHp37A+CEhv1C6jYZ0QQnQumQnQKwqc3awdidkO2qom1ukmGyp1r1Jqn1Jqn9yoXAjRodQbIOsA\nBMVaOxKg/Qk/58JUTcMyt2F9BhDSqF0wkNXUDrTW72qtY7XWsX5+fu0MRwghbEheEtSUQ3DHSPjr\ngIUNjxcCaxutX9Bwtk4cUHJh6kcIITqNFh6w1brJCRCTa81pmZ8Cu4BIpVSGUupu4AVgmlLqFDCt\n4TnABuAMkAz8G/idSaMWQgh7kLkP3LqBd5/LNtFaM/aFzbyx6ZTZw2nxjRW11rdf5qUpTbTVwP1t\nDUoIITqEjAQIHA4Olx9b55VVk11ShZeb+e9zK1faCiGEOdRUQO6xK87fJ+WUAdC/V1ezhyQJXwgh\nzCH7EGjDFefvk84ZE35kT0n4Qghhn1p4wDbpXBm+nq74eJr/TliS8IUQwhwy9kK3UPD0b7bZyZwy\nInt5WiQkSfhCCGFqWkNaPITGNdusvl5zMqecyJ5eFglLEr4QQphaUQqU51wx4acXVXK+1iAjfCGE\nsFtp8cblFRL+iYYDtv0tcMAWJOELIYTppe0C127gN7DZZkmS8IUQws6lxUPo6GYvuAI4nl1KuI87\nHq7mv+gKJOELIYRpVRZCftIVp3PAmPAHBljmgC1IwhdCCNNK321chjSf8Mur60gpqGSQJHwhhLBT\nabuMd7gKav5W3knnSgFkhC+EEHYrLR4Ch4Fzl2abHcsyJvxBgZLwhRDC/tRWGe9w1YL5+2PZZXTr\n4kxAN8vd+lASvhBCmErWATDUQOiYKzY9ll3KoAAvlGrqjrDmIQlfCCFMJXW7cRkyutlmhnpN0jnL\nnqEDkvCFEMJ0zm6DnlHg4dN8s/wKqmrrLTp/D5LwhRDCNOqqjadk9p5wxabHshsO2NrjCF8p9bBS\nKlEpdVQp9alSyk0p1VsptVspdUoptUop5WKKvoQQwiZl7IW6Kgi/6opNj2eX4uyo6OtvmaJpF7Q7\n4SulgoAHgVitdRTgCNwGvAi8prXuBxQBd7e3LyGEsFlnt4FygLCxV2x6LKuUvv5dcXGy7CSLqXpz\nAroopZwAdyAbuBpY3fD6h8ANJupLCCFsz9mtEDAUunS/YtNj2aUMDLBMwbTG2p3wtdaZwCtAGsZE\nXwIkAMVa67qGZhlAUFPbK6XuVUrtU0rty8vLa284QghheTWVximdFkzn5JRWkVdWTXRQNwsE9kum\nmNLpAcwBegOBgAdwTRNNdVPba63f1VrHaq1j/fz82huOEEJYXno81NdC74lXbHo4owSAIcF2mPCB\nqcBZrXWe1roW+BIYC3RvmOIBCAayTNCXEELYnrPbwMGpRVfYHskoxkHBoAD7TPhpQJxSyl0ZLxmb\nAhwDtgA3NbRZCKw1QV9CCGF7UrZB4HBwvfJZN4czS+jfsytdXBwtENgvmWIOfzfGg7P7gSMN+3wX\neBx4RCmVDPgA77e3LyGEsDlVpZC5v0Xn32utOZpZYpX5ezCeXdNuWus/AX+6ZPUZYJQp9i+EEDYr\nZRtoA0Rcef4+u6SK/PIaoq0wfw9ypa0QQrRP8iZw8bziDU/g5wO21hrhS8IXQoi20hqSNxqnc5yu\nXEzgSGYxTg7K4kXTLpCEL4QQbVV4BopToc/VLWp+OMN4wNbN2fIHbEESvhBCtF3yRuOy79QrNtVa\ncySzxCrn318gCV8IIdoqeSN4R4B37ys2zSg6T3FlrdUO2IIkfCGEaJvaKkjZ3qLRPcChjGLAegds\nQRK+EEK0TdouqK2EPlNa1PxAWjGuTg4M6GWdA7YgCV8IIdrm9CZwdIHw8S1qfiCtiOigbhYvidyY\nJHwhhGiL5E3G2jktKKdQXWfgaGYpw8N6WCCwy5OEL4QQrVWUArnHoP/MFjVPzCqlxlDP8NAr18o3\nJ0n4QgjRWknfGJeRTVWC/7UDacYDtsNCZYQvhBD2JWkD+A0wnpLZAvvTigjq3oWeXm5mDqx5kvCF\nEKI1zhdByg6InNXiTQ6kFjHMytM5IAlfCCFa59RGY3XMFib8cyVVZJVUWX06ByThCyFE6yRtAA9/\nCBrRouYH0ooArH7AFiThCyFEy9XVGMspRM4Eh5alzwPpxbg4OTA40HpX2F4gCV8IIVoqdQdUl7Zq\n/n5/ahFRgV5WveDqApNEoJTqrpRarZQ6oZQ6rpQao5TyVkr9oJQ61bC0/gSWEEK0R9IGcOoCva98\ndyuAqloDhzNKGBnubebAWsZUHzl/B77VWg8AhgLHgSeATVrrfsCmhudCCGGf6g1wbB30mwYu7i3a\n5GB6MTWGekb17iAJXynlBUyg4SblWusarXUxMAf4sKHZh8AN7e1LCCGsJi0eys/B4Jansj1nC1EK\nYsM6SMIHIoA8YKlS6oBS6j2llAfQU2udDdCw9DdBX0IIYR2Ja4zTOf1mtHiTPWcLGdDLi27uzmYM\nrOVMkfCdgOHA21rrYUAFrZi+UUrdq5Tap5Tal5eXZ4JwhBDCxOoNcGytcTqnBcXSAGoN9SSkFjEq\n3HYOX5oi4WcAGVrr3Q3PV2P8AMhRSgUANCxzm9pYa/2u1jpWax3r5+dngnCEEMLE0nZBRS4Mntvi\nTRKzSjlfa2BUbx8zBtY67U74WutzQLpSKrJh1RTgGLAOWNiwbiGwtr19CSGEVVyYzunfmumcAgBG\n9radEb6TifbzAPCxUsoFOAMsxvhh8plS6m4gDbjZRH0JIYTlXJjO6T8dXDxavNmes4VE+Hrg39W6\nBdMaM0nC11ofBGKbeKll9/4SQghblboDKvJaNZ1TX6/Zc7aQWdEBZgys9ax/6ZcQQtiyw6vAxRP6\nTW/xJkk5ZZRW1dnM+fcXSMIXQojLqamExLUwaE6rpnN2njbO34+OsJ0DtiAJXwghLi9pA9SUwdDb\nWrXZjuR8Inw9COrexUyBtY0kfCGEuJxDK8ErGMLGt3iTWkM9u88UMLavbY3uQRK+EEI0rSwHTm+C\nIbe0uBQywKH0YipqDIzv62vG4NpGEr4QQjTlyOeg61s9nbM9OR+lIM7G5u+hgyR8rTVVtQZrhyGE\n6EgOr4TA4eAXeeW2jexMLiA6qBvd3V3MFFjbdYiEv+t0AWP+tomXvj1Bdsl5a4cjhLB32Yfg3JFW\nj+4rquvYn1bEOBuczoEOkvC7uTszMtybd346zfgXt3D/J/vZl1KI1traoQkh7FHCMnByM87ft8Ke\ns4XU1WvG9bHNhG+q0gpWNTiwG+8uiCW9sJKP4lNZuSeN9YeziQ7qxqKx4Vw3NABXJ0drhymEsAfV\n5XD4c+OVtV1aVwdnR3I+Lk4OxNpQhczGOsQI/4IQb3eenDWQ+Cen8Jcbojhfa+APnx9i3AubefX7\nJHJLq6wdohDC1h39wnju/YhFrd70p5N5jAzvgZuzbQ4wO8QI/1LuLk7Mjwtj3uhQtifns2xHCv/Y\nksw/fzzNtUMCWDQ2nGGhtvkJLISwsoRl4DcQQka3arOMokpO5ZZz68gQ88RlAh0y4V+glOKqfn5c\n1c+PlPwKPtyVwuf7Mlh7MIuYkO4sHhfONVEBNnE3eSGEDcg+BFn7YeaLoFSrNv0xyXgDp8kDbPfm\nfp0m04X7evCn6wcT/+QUnp09mJLztTy08iDjX9zMG5tOkVdWbe0QhRDWduFg7dBbW73pj0m5hHh3\nIcK35TV3LK3TJPwLPF2dWDg2nE2PTGTp4pEMDPDi1R9OMu6FzTzy2UGOZpZYO0QhhDWcL4ZDq2Dw\nja0+WFtVa2BHcgGTI/1RrfxmYEkdekqnOQ4OismR/kyO9Cc5t5zlu1JYnZDBl/sziQ3rwaJx4cwY\n3Atnx073mShE53TgI6itgLj7Wr3pnrOFnK81MDnSdqdzoBMn/Mb6+nvy3JwoHp0RyWd701m+K5X/\n+eQAAd3cmB8Xxu2jQvH2sL2r5oQQJmKog93vGoukBQxt9eZbknJxdXKwyXIKjZls+KqUclRKHVBK\n/afheW+l1G6l1Cml1KqG2x/aNC83Z+65KoItj07ivQWxRPh58PJ3SYz52yYeX32Y49ml1g5RCGEO\nJ/4DJWkQ999t2vynpDzG9PGhi4ttno55gSnnKx4Cjjd6/iLwmta6H1AE3G3CvszK0UExdVBPPr4n\nju8fnsBvRgSz9lAm1/x9G7f+axffHj2HoV6u4hWiw4h/G3qEQ+Q1rd70bH4FZ/IrbH46B0yU8JVS\nwcC1wHsNzxVwNbC6ocmHwA2m6MvS+vfsyvNzo4lfMoUl1wwgo+g8961IYMJLW/jXT6cpqay1dohC\niPbITID0eBh9Hzi0foT+XeI5AKYO6mnqyEzOVCP814E/AvUNz32AYq11XcPzDCDIRH1ZRXd3F/5r\nYh9+emwS78wfTnCPLvztmxPE/W0TT645wsmcMmuHKIRoi51vgktXiJnXps2/SzxHdFA3m7u7VVPa\nfdBWKXUdkKu1TlBKTbqwuommTc6BKKXuBe4FCA0NbW84Zufk6MDMqABmRgVwLKuUZTvPsjohg092\npzG+ry+LxoYzeYA/jg62e2qWEKJBfjIkroFxD4GbV6s3zymt4kBaMX+Y1t8MwZmeKUb444DZSqkU\nYCXGqZzXge5KqQsfKMFAVlMba63f1VrHaq1j/fz8TBCO5QwK9OKlm4YSv2QKj82IJDm3nHuW72Py\nKz/y3rYzlFbJdI8QNm37a+DkCmPub9Pm3x/LAWBGVC9TRmU27U74WuslWutgrXU4cBuwWWs9D9gC\n3NTQbCGwtr192SpvDxfun9yXbY9P5s07huHf1ZW/rD9O3PObeHrtUU7nlVs7RCHEpYrTjTc5Gb4Q\nPNt2wPX7xHP09vWgn7+niYMzD3Oeh/84sFIp9RfgAPC+GfuyCc6ODlw3JJDrhgRyJKOEpTvPsnKP\n8bz+if39WDQunIn9/HCQ6R4hrG/nG4CCcQ+2afOSylp2nS7g7qt62/TVtY2ZNOFrrX8Efmx4fAYY\nZcr925M4w+0LAAAaXElEQVTo4G68eksMS64ZyCe701ixO5XFS/cS4evBwrHh/GZEMJ6uct2bEFZR\nngv7lxvvaNUtuE272JyUQ129ZsZg+5jOgU5YS8fS/Lq68tDUfux4/Gr+flsMXbs486d1icQ9v4ln\nv04kJb/C2iEK0flsexUMtTD+4TbvYsORc/T0ciUmuLsJAzMvGWJaiIuTA3NigpgTE8SBtCKW7Uzh\no12pLNuZwtWR/iwaF874vr5289VQCLtVnA773oeYO8CnT5t2UXK+lp+S8rhzTJhdTdFKwreCYaE9\nGBbagydnDeTj+FQ+3p3Gne/voa+/J4vGhnPj8CDcXeSfRgiz+OkF43LSE23exXdHz1FjqOf6oYEm\nCsoyZErHinp6ufHI9Eh2PHE1/3fzUNycHfh/Xx0l7vlNPL/hOOmFldYOUYiOJf8UHPwERt7T5rl7\ngK8PZxHm487Q4G4mDM78ZBhpA9ycHfnNiGBuHB5EQmoRS3em8P72s7y37QxTB/Zk8bjexEV4y3SP\nEO21+S/g1AXGP9LmXeSVVbMjOZ/fTeprd/8nJeHbEKUUseHexIZ7k1V8nhXxqXy6J43vj+UwoFdX\nFo0N54ZhQTZ7g2QhbFrGPjj2FUx4DDzbfpHnhiPZ1GuYHWNf0zkASmvbqfoYGxur9+3bZ+0wbEpV\nrYG1BzNZuiOFE+fK6O7uzO2jQrkzLoxAO6jdIYRN0Bremwol6fBAArh2bfOubnp7J2VVdXz38AQT\nBtg+SqkErXXsldrJCN/GuTk7cuvIUG6JDWH32UKW7jjLv346zbtbzzBzcC8WjQsnNqyH3X21FMKi\njnwOmftgzlvtSvZpBZXsSy3isRmRJgzOciTh2wmlFHERPsRF+JBeWHlxumf9kWwGB3qxeFxvrhsS\nINM9QlyqpgJ++BMExMDQO9q1q9UJ6TgouHG4fRb/lbN07FCItztLZg0k/skp/HVuFDV19Tz6+SHG\nvbCZ//s+iZzSKmuHKITt2PF3KMuCa14Eh7anPEO9ZnVCBuP7+RHQzT6nUyXh2zF3FyfmjQ7j+4cn\nsOLu0QwL7c6bW5IZ98JmHvz0APvTiqwdohDWlZ8M21+HqJsgNK5du9p5Op+skipuHtH20zmtTaZ0\nOgClFOP7+TK+ny+pBRV8uDOVz/els+5QFkNDurN4bDizogNwcZLPd9GJaA3/+T04ucGM59u9u8/3\nZdCtizPT7ODOVpcjGaCDCfPx4OnrB7HrySk8N2cwZedr+f2qg4x7cTOvbzxJXlm1tUMUwjIOfQop\n22DaM9C1fUm6pLKWbxPPMScm0K6Pk8kIv4PydHViwZhw5o8OY+upPJbtTOH1jaf455bTXDckgEXj\nwhliR0WfhGiVigL47ikIiYPhi9q9u3WHMqmpq+fmESHtj82KJOF3cA4OikmR/kyK9Od0XjnLd6aw\nOiGDLw9kMiKsB4vGhjMzqhfOjvJlT3Qg3/wRqkvh+tfbdaAWQGvNR/GpRAV5ERXU+tsg2hL5X96J\n9PHz5Nk5Uex6cgr/e90g8sqqeeDTA1z14hbe2pJMQblM94gO4OiXcHQ1THwc/Ae2e3e7zxZyMqec\nBXHhdn+9i1xp24kZ6jVbTuSybGcK25PzjSWchwayaFw4gwPtqyiUEACUnYN/xoF3BNz1PTi2fxLj\n/o/3sz05n/glU+jiYpvz93KlrbgiRwfF1EE9mTqoJydzyli2M4Uv92fweUIGo3p7s3hsONMG9cRJ\npnuEPdAa1j0Atedh7r9MkuxzSqv4LvEci8eF22yyb412/09WSoUopbYopY4rpRKVUg81rPdWSv2g\nlDrVsOzR/nCFufTv2ZXn50YTv2QKT84aQGbRef774/1MfPlH3vnpNEUVNdYOUYjm7X0PTn0P054D\n334m2eWne9IwaM38uDCT7M/a2j2lo5QKAAK01vuVUl2BBOAGYBFQqLV+QSn1BNBDa/14c/uSKR3b\nUWeoZ+PxXJbtPEv8mUJcnRy4ISaIBWPDZLpH2J6sA/D+dIiYDLevbPeBWoDqOgPjX9zC4EAvli22\n7dtzW2xKR2udDWQ3PC5TSh0HgoA5wKSGZh9ivLl5swlf2A4nRwdmRvViZlQvjmeXsnxXCmsOZLJq\nXzojw3uwYIyc3SNsxPli+GwhePjD3HdMkuwB1h7IIq+smnvGR5hkf7bApAdtlVLhwFYgCkjTWndv\n9FqR1vpX0zpKqXuBewFCQ0NHpKammiweYVollbV8ti+d5fEppBeex7+rK/NGh3H76BD8u7pZOzzR\nGWkNny2ApA2w+BsIMc1IvL5eM+21n3B1cmT9g+Nt/uyclo7wTTY8U0p5Al8Av9dal7Z0O631u1rr\nWK11rJ9f229KIMyvm7szv50QwY+PTub9hbEMCPDitY0nGffCZh5aaazdY0tnfYlOYNsrcHwdTH3G\nZMkeYPOJXE7nVfBfEyNsPtm3hknO0lFKOWNM9h9rrb9sWJ2jlArQWmc3zPPnmqIvYX2ODoopA3sy\nZWBPzuSVs3xXKqsTMlh7MIvooG4sHBsupZqF+R3/2njLwiG3wpj/Memu/7X1NEHdu3BtdIBJ92tt\npjhLRwHvA8e11q82emkdsLDh8UJgbXv7ErYnws+TZ2YPJv7JKfx5zmDO1xp49PNDjH1hMy99e4LM\n4vPWDlF0ROeOwJf3QlAsXP8GmHAUnpBayN6UIu4e37vDnZJsirN0xgPbgCNAfcPqJ4HdwGdAKJAG\n3Ky1LmxuX3KWjv3TWrPzdAEf7kxh4/EcAKYP6sWCsWGMifDpUF+PhZWUZsP706DeAPduga69TLr7\nO9/fzbGsUrb+cTIervZxqZIlz9LZDlzuf/GU9u5f2BelFOP6+jKury/phZV8vDuNlXvT+DbxHP17\nerJgTDhzhwXZzX8kYWPOF8GKG43LRetNnuz3phSy7VQ+T80a2CH/RqW0gjC7qloD6w5l8eHOFBKz\nSunq5sTNI0JYMCaMcF8Pa4cn7EVNJXw0F7L2w7zPIWKSybu449/xnMwpZ9sfJ9vVlbVSWkHYDDdn\nR26JDeHmEcHsTyviw52pLN+Vwgc7zjIp0o+FY8OZ2M8PBweZ7hGXYaiF1YshfTfcvNQsyT7+TAE7\nTxfwv9cNsqtk3xoywhdWkVtaxSd70vh4dxp5ZdWE+7hz55hwbhoRTLcuztYOT9iSuhpjsj/xH7j2\n/2DkPSbvQmvNLf/aRWpBJVv/ONnuzjCz+Hn4QrSGv5cbv5/anx2PX80btw/D19OVP//nGGP+tomn\n1hwh6VyZtUMUtqCuGj5faEz217xklmQP8F1iDntTinhoaj+7S/atISN8YTOOZpawfFcKaw9mUV1X\nT1yEN4vGhjN1oFTs7JRqq4zJ/uS3MOsVGPVbs3RTU1fP9Nd+wtnRgW8eusou/9ZkDl/Ynaigbrx0\n01CWXDOQVfvS+WhXKvet2E9ANzduHxXKbSND8PeSEg6dwvki+PQOSNsJ174KI+82W1cr4lNJKahk\n6eKRdpnsW0NG+MJmGeo1G4/nsCI+lW2n8nFyUMwY3Iv5cWHERXjLOf0dVUkmrPgNFCQbi6FF32S2\nroora5j0yo9EB3Vj+V2j7PZvSkb4wu45NiT4GYN7cTa/gk92p/LZvgzWH8mmr78n80eHcuOIYLzc\n5CBvh5F9GD69DapKYf4XEDHRrN29+G0SZVV1PDlroN0m+9aQEb6wK1W1Bv5zOJuP4lM5lF5MF2dH\nbhgWyPw4qdNv946shrX/A+7expr2AUPM2l1CahG/eXsnv72qN09dO8isfZlbS0f4kvCF3TqSUcKK\n+FTWHsqkqraeYaHduTMujFnRUrjNrtQbYNOzsOPvEDoGblkOnv5m7bLOUM91/9hOyflaNj4y0e6v\nqpWELzqNkspavtifwYr4VM7kV9DD3ZlbYkO4Y3QoYT5yJa9NK80yFkFL2Qaxd8PMF8DJxezdvrv1\nNM9vOME780cwM8q05RmsQRK+6HQuFG5bEZ/K98dyqNeaCf38uDMujMkD/HGUK3lty4n1sPZ+47n2\ns16GYfMt0m1ybhmz3tjOhH5+/HvBiA4xdy8HbUWn07hw27mSKj7dYyzcds/yfQR178Ido0O5JTYE\nv66u1g61c6sugx+ehn0fQK8hcNMHJrvp+JXUGup55LNDeLg48vyNUR0i2beGjPBFh1ZrqGfjsRxW\n7E5lR3IBzo7GM3/uGBVKXISP1O+xtJPfw38ehtJMGHM/THkanCz3Afz6xpO8vvEU/5w3nFkd6OYm\nMsIXAnB2dOCa6ACuiQ7gdF45K+JT+XJ/Jv85nE24jzu3jgzlphHBMuo3t4p8+HYJHPkMfCPh7u9N\nekvClkhILeTNzcnMiQnsUMm+NWSELzqdqloD3xzN5tM96ew5W4iTg2L64J7cPiqUcX18ZdRvSnU1\nsOdf8NPLUFsJV/0BrnrEoqN6gILyaq59YzsuTg58/cD4DlegT0b4QlyGm7Mjc4cFM3dYMMm55azc\nk8YX+zPYcOQcId5duG1kKDePCJYyDu2hNSRtgO//HxSegX7TYfpfwC/S4qHU12se/uwQhZU1fPnf\nYztcsm8Ns4/wlVIzgb8DjsB7WusXLtdWRvjCWqrrDHyXmMOnu9PYdabAeKP2Af7cPjqUCf385Ayf\nltIaTm+CLX+DzH3G6ZsZz0O/qVYL6dUfTvLGplP8dW4U80aHWS0Oc7KJEb5SyhF4C5gGZAB7lVLr\ntNbHzNmvEK3l6uTI7KGBzB4ayNn8ClbuTWP1vgy+P5ZDUPcu3BIbwi0jgwno1sXaodomreHMFmOi\nz9gD3ULguteNp1o6Wm9EvfZgJm9sOsVNI4K5Y1So1eKwFWYd4SulxgDPaK1nNDxfAqC1/ltT7WWE\nL2xJTV09PxzLYeXeNLadysdBwaRIf24eEcyUgT1xcerYlRVbpK4ajn4B8f+Ec0fAK8g4Tz9svsXn\n6S+VkFrE7f+OJyakOyvuHt2h/71sYoQPBAHpjZ5nAKPN3KcQJuHi5MC1QwK4dkgAaQWVrNxrnOvf\nfCKXHu7OzIkJ4qYRwUQFdcIaPmXnIOFD2PseVOSC3wC4/u8w9HarJ3qA5Nxyfrt8HwHd3PjX/BEd\nOtm3hrkTflMTn7/4SqGUuhe4FyA0VL5yCdsU6uPOH2cO4A/TI9l2Ko/PEzL4ZHcay3amMDDAi5tG\nBHNDTCA+ntZPdmZjqIWT38GBFXDqe9AG6DsV4n4Hfa4GG7mIKb2wkvnv7cZBKZYtHkUPD/OXarAX\nMqUjRBsVV9bw9aEsPk/I4HBGCU4OiqsH+HNzbAiTIv1w7gg306ivN87JJ34FR1dDRR549jSO5IfN\nt9gVsi11rqSKW/61i5Lztay8N46BAV7WDskibKKWjlLKCTgJTAEygb3AHVrrxKbaS8IX9irpXBmr\nE9JZcyCT/PIafD1duCEmiN+MCLa/pFNvgPTdcGwtHFsHZVng6Ar9psGwO42jekfbO6M7raCSee/H\nU1hew4p7RjMstIe1Q7IYm0j4DYHMAl7HeFrmB1rrv16urSR8Ye9qDfX8mJTH6oR0Nh3Ppa5eE9mz\nK7NjApkTE0hwD3drh9i0shzj6ZSnfoDTm6Gq+OckP+gG6D8D3Gz3g+tUThnz3ttNjaGeZYtHERPS\n3dohWZTNJPzWkIQvOpKC8mo2HMnmq4NZJKQWARAb1oM5w4K4NjoAb2vOLZfnQdou40/KNuMZNmCc\nruk71fjTbxq4drVejC20Izmf/16RgKuzIyvuHk1kL9uP2dQk4QthQ9ILK1l3KIuvDmRyKrccJwfF\nxP5+zI4JZNqgnri7mHGKxFALeUmQfRDS9xiTfP5J42tObhA8EiImGRN8z2hwsJ9jDx/Fp/LMukT6\n+Hnw/sKRhHjb6DcoM5OEL4QN0lpzPLuMtQczWXcoi+ySKtxdHLl6gD+zogOYFOnXvuRfXWZM5ueO\nGhN89iHjY0O18XXXbhAaB2FjIHQsBA6zyA1HTK2ypo7nvj7Gyr3pXD3An7/fFkPXTnxvY0n4Qti4\n+nrNnpRC1h7M4vvEcxRU1ODm7MCk/v5cE92Lqwf4N53E6uuh/BwUnoX8JMg7CXknjIm+NPPndq5e\nEDDU+BM4zLj07mNXI/imHMsq5YFP93Mmv4L/ntiHP0yP7PSlLyThC2FHDPWaPWcL+eZoNpuPpEFF\nLoGOJUwJqGasTwX9XYtwLU+HolQoSQdDzc8bO3sYT4/0iwTf/sal/yDo0dvuk3tj1XUG/vXTGd7c\nnEx3d2devzWGsX19rR2WTbCVK22FEPX1UF0ClYVwvsj4U1kIlQXGkXpZDo7l5xhTlsOYsmyeqyuG\nC9dv5Rl/CrUnGS4B0CMC7+jpdA/si/IONxYn8wrqUIm9KfFnCnhqzRFO51Vw3ZAAnpsTZd2D3nZK\nEr4QTdHaOIquPW+sF1NTbvypvrAsM/5cbt35YjhfaEzsVcWg65vux8EZuvYynh3j0wfCx4FnL+ja\nEzx7Ud81gCOV3fk+uYJNx3M5kVYGaRDq7c7VA/yZMtCNUZ4a1w6a75POlfHyd0lsPJ5DiHcXli0e\nyaRIf2uHZbdkSke0nNYNPwZjAmvqp/4y61vyU29o2P+F53VQXwuGC8vahnV1DY+bWnfpa4afHxtq\noK4KaquMy7qqnxN6XcPy4vMqLqkC0jxnd3DxBFdP47JLD3D3Ni67eDc89m603vvn11tRkiCz+Dyb\nT+Sy5UQuO5Lzqa6rx93FkVG9vRnXx5exfX0Y2MvL7m/iciSjhH9vO8PXh7PwdHXivol9uGtcb7q4\nOFo7NJvUuebwkzfCd08ZH198P43e16XrfvGeW7uOZto106ep17U6tpauu6TPxgm5NQnQ2hycjWV5\nHZyMP47OxguJnN2Mxb2cuhiXzl1a9tzZw5jMXbuCS9efE/uFpYPlE9H5GgM7T+fzY1IeO07ncyav\nAgBvDxfGRPgwpo8PcRHeRPh62sUHQHWdgc3Hc1m2M4XdZwvxcHFkflwY903sI/VwrqBzzeG7el1y\nJ52GP+5fjJwuXdfoNVOu+8X/q5bE0Z517YmXy7dr3EY5gnL49Y9DE+ta+uPgaOzjsm0cjaE4OP0y\ncTs6G587OBkv7W8qqTs4/7z/Dq6LiyNTBvZkysCeAGSXnGdncgE7TuezM7mA9UeyAfByc2JYaA9G\nhPVgeGgPhoZ0s5lTGGsN9SSkFrHuUBbrD2dTcr6WwG5uPDVrILeOCsHLRuLsKDrGCF8I8Qtaa87m\nV7AvtYj9qUXsTyviVG45Whs/C3v7eDAgoCsDe3kxMMCLAQFdCereBWXmD8o6Qz2ncss5kFbM9uQ8\ntp3Mp6y6ji7OjswY3JMbhgUxvq8vTh2h8JwFda4RvhDiF5RSRPh5EuHnyS2xIQCUVtVyMK2YA2nF\nHM8uJTGrlA1Hzl3cxs3ZgVBvd0K9PQjzcSfMx52eXm74errg6+mKj6crHi6OV/xQqKmrp6iyhvzy\narKLq0gpqOBMfgXJOeUczSqhssYAQC8vN64bGsDE/v5c1c8XD1dJR+Ymv2EhOgkvN2cm9PdjQn+/\ni+vKq+tIOlfG8exSUvIrSC2sJK2gku3JeVTV/vrMImdHhZuTI67Ojrg5O+Di6IBBa2rr6qkxaKpr\nDZRV1/1qux7uzvT29eCW2BBiQroTE9KdMB93s3+jEL8kCV+ITszT1YkRYcb5/ca01uSVV5NbWk1+\neTUF5cYRe/H5WqpqDVTX1VNVa6Cmrh4nB4WzowPOTsYPAG8PF3w8XfDxcKWnlyu9fT3o7i4HXW2B\nJHwhxK8opfDv6oZ/VzdrhyJMSI6MCCFEJyEJXwghOglJ+EII0Um0K+ErpV5WSp1QSh1WSq1RSnVv\n9NoSpVSyUipJKTWj/aEKIYRoj/aO8H8AorTWQzDerHwJgFJqEHAbMBiYCfxTKSVFMIQQworalfC1\n1t9rrS+cdBsPBDc8ngOs1FpXa63PAsnAqPb0JYQQon1MOYd/F/BNw+MgIL3RaxkN64QQQljJFc/D\nV0ptBHo18dJTWuu1DW2eAuqAjy9s1kT7Jov2KKXuBe4FCA0NbUHIQggh2uKKCV9rPbW515VSC4Hr\ngCn650psGUBIo2bBQNZl9v8u8G7DvvKUUqktiNvW+AL51g7CwuQ9dw7ynu1DWEsatataplJqJvAq\nMFFrnddo/WDgE4zz9oHAJqCf1trQ5s5smFJqX0sq1XUk8p47B3nPHUt7Syu8ifHumz80FEGK11rf\np7VOVEp9BhzDONVzf0dN9kIIYS/alfC11n2bee2vwF/bs38hhBCmI1famsa71g7ACuQ9dw7ynjsQ\nm7rjlRBCCPOREb4QQnQSkvBNTCn1qFJKK6V8rR2LuTVXS6kjUUrNbKgJlayUesLa8ZibUipEKbVF\nKXVcKZWolHrI2jFZilLKUSl1QCn1H2vHYg6S8E1IKRUCTAPSrB2LhTRZS6kjaagB9RZwDTAIuL2h\nVlRHVgf8QWs9EIgD7u8E7/mCh4Dj1g7CXCThm9ZrwB+5zFXFHU0ztZQ6klFAstb6jNa6BliJsVZU\nh6W1ztZa7294XIYxAXb40ihKqWDgWuA9a8diLpLwTUQpNRvI1FofsnYsVtK4llJH0qnrQimlwoFh\nwG7rRmIRr2McsP367u0dhNzTthWaqysEPAlMt2xE5tfGWkodSYvrQnU0SilP4Avg91rrUmvHY05K\nqeuAXK11glJqkrXjMRdJ+K1wubpCSqlooDdwqOGK42Bgv1JqlNb6nAVDNLk21lLqSFpcF6ojUUo5\nY0z2H2utv7R2PBYwDpitlJoFuAFeSqkVWuv5Vo7LpOQ8fDNQSqUAsVpreyvA1CqXq6XUkSilnDAe\nkJ4CZAJ7gTu01olWDcyMlHHU8iFQqLX+vbXjsbSGEf6jWuvrrB2LqckcvmiPN4GuGGspHVRKvWPt\ngEyt4aD0/wDfYTx4+VlHTvYNxgF3Alc3/LsebBj5CjsnI3whhOgkZIQvhBCdhCR8IYToJCThCyFE\nJyEJXwghOglJ+EII0UlIwhdCiE5CEr4QQnQSkvCFEKKT+P80IDiomdKT7QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x111009940>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = gluon.loss.Poisson()\n",
    "with ag.record():    # start recording\n",
    "    theloss = loss(output, 10 * nd.ones_like(output))\n",
    "theloss.backward()   # and compute the gradient\n",
    "\n",
    "plt.plot(output.asnumpy(), theloss.asnumpy(), label='Poisson loss')\n",
    "plt.plot(output.asnumpy(), output.grad.asnumpy(), label='Gradient of Poisson loss')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "[ 7.53516912  0.          0.          0.          6.3003993 ]\n",
      "<NDArray 5 @cpu(0)>\n"
     ]
    }
   ],
   "source": [
    "# this implements an L2 norm triplet loss\n",
    "# max(margin + |f1 - f2|^2 - |f1-f3|^2, 0) per observation\n",
    "def TripletLoss(f1, f2, f3):\n",
    "    margin = 1\n",
    "    loss = nd.sum((f1-f2)**2 - (f1-f3)**2, axis=1) + 1\n",
    "    loss = nd.maximum(loss, nd.zeros_like(loss))\n",
    "    return loss\n",
    "\n",
    "loss = TripletLoss\n",
    "#with ag.record():    # start recording\n",
    "#    theloss = loss(output, nd.ones_like(output))\n",
    "#theloss.backward()   # and compute the gradient\n",
    "#plt.plot(output.asnumpy(), theloss.asnumpy(), label='Huber Loss')\n",
    "#plt.plot(output.asnumpy(), output.grad.asnumpy(), label='Gradient')\n",
    "#plt.legend()\n",
    "#plt.show()\n",
    "\n",
    "f1 = nd.random_normal(shape=(5,10))\n",
    "f2 = nd.random_normal(shape=(5,10))\n",
    "f3 = nd.random_normal(shape=(5,10))\n",
    "\n",
    "theloss = loss(f1, f2, f3)\n",
    "print(theloss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Classification \n",
    "## Logistic Regression\n",
    "\n",
    "Next consider the case where we have two labels, say ``cat`` and ``dog``. Since statisticians (and computers) don't like strings, we simplify this to $y \\in \\{\\pm 1\\}$. One way of mapping real numbers in $\\mathbb{R}$ into class probabilities is to use a sigmoid function.\n",
    "\n",
    "$$p(y|f) = \\frac{1}{1 + \\exp(-y f)} \\text{ and hence } -\\log p(y|f) = \\log(1 + \\exp(-y f))$$\n",
    "\n",
    "*Side remark for math nerds:* To keep the term numerically stable we can rewrite it as $-yf + \\log(1 + \\exp(yf))$ whenever $yf < 0$. The reason for doing this is to avoid exponentiating a large positive number which would trigger a numerical overflow.  Combining both expressions we get the following expression: $\\log(1 + \\exp(-|yf|)) - \\delta(yf < 0) \\cdot yf$. As we can see, the probabilities converge to 0 and 1 respectively for extreme values of $f$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlYVGX/x/H3GQYYdhRQUUTUEhcYFpHF3dzK1FIzSxOx\n1MxSW/Sn1ZOltmhPmWkuuZuZ2qb5pGXuS6IIigsG7rgim+wMs53fH+jkAq4Dw3K/rmuuQebMOd8h\n+nDPmft8b0mWZQRBEISqQ2HpAgRBEATzEsEuCIJQxYhgFwRBqGJEsAuCIFQxItgFQRCqGBHsgiAI\nVYwIdkEQhCpGBLsgCEIVI4JdEAShilFa4qDu7u6yj4+PJQ4tCIJQacXFxaXLsuxxr+0sEuw+Pj7E\nxsZa4tCCIAiVliRJyfeznTgVIwiCUMWIYBcEQahiRLALgiBUMRY5xy6ULZ1Ox8WLF9FoNJYuRahG\nVCoVXl5eWFtbW7qUak8EexV08eJFnJyc8PHxQZIkS5cjVAOyLJORkcHFixdp2LChpcup9swS7JIk\nnQNyAQOgl2U5xBz7FR6ORqMRoS6UK0mScHNzIy0tzdKlCJh3xN5JluV0M+5PeAQi1IXyJn7nKo5K\n9eHpofPXmL/ztKXLEARBeCixKbGUx3Kk5gp2GfhLkqQ4SZJGlLSBJEkjJEmKlSQp9mHfrq09dIlp\nfySyeM/ZR6lVKAeOjo6PvI/Lly/z3HPPlfp4VlYWc+fOve/tbxcVFcXPP//8SDUKwv36/vj3DN00\nlE3nNpX5scwV7G1kWQ4GngJelySp/e0byLK8QJblEFmWQzw87nlFbIkm9WzOU351mPr7cX6MvfCI\nJQsVXd26de8avLcH+722FwRLWXdqHdMPTKeLdxe6NOhS5sczS7DLsnz5+n0qsBYINcd+b6e0UjDz\nhUDaPe7OxF+O8OexK2VxGKGMJCcn07lzZ9RqNZ07d+b8+fMAnD59mvDwcFq1asWkSZNMo/1z587h\n5+cHQEJCAqGhoQQGBqJWqzl58iQTJ07k9OnTBAYGMn78+Fu2NxgMjBs3Dn9/f9RqNbNnz75rbVu3\nbiUoKAh/f39efvllioqKAJg4cSLNmzdHrVYzbtw4AH766Sf8/PwICAigffs7xjCCcIutyVv5cO+H\nhHuGM739dJSKsp+M+MhHkCTJAVDIspx7/etuwJRHrqwUtkorvh3cksGLYxizKp7FUUraPf5w7wCq\ng8n/S+D45Ryz7rN5XWc+7NXigZ/3xhtvEBkZyZAhQ1iyZAljxoxh3bp1jB07lrFjx/Liiy8yf/78\nEp87f/58xo4dy6BBg9BqtRgMBqZNm8axY8eIj48Hiv8Q3LBgwQLOnj3LoUOHUCqVZGZmllqXRqMh\nKiqKrVu30qRJEyIjI5k3bx6RkZGsXbuWxMREJEkiKysLgClTprBp0ybq1atn+p4glCT6cjTjd43H\nz92Przt9jY2VTbkc1xwj9trAHkmSDgMxwAZZlv80w35LZW+jZMmQVjTycGDEd3HEJV8ry8MJZhId\nHc3AgQMBGDx4MHv27DF9v3///gCmx28XERHBp59+yvTp00lOTsbOzu6ux9qyZQsjR45EqSweu9Ss\nWbPUbZOSkmjYsCFNmjQBYMiQIezatQtnZ2dUKhXDhg3j119/xd7eHoA2bdoQFRXFwoULMRgMD/AT\nEKqTw2mHGbt9LD4uPsztPBd7a/tyO/Yjj9hlWT4DBJihlgfiYm/NilfC6D9/L0OXxrDm1QiaeTqX\ndxkV3sOMrMvLg0yPGzhwIGFhYWzYsIHu3buzaNEiGjVqVOr2sizf9/5Lm6WgVCqJiYlh69atrF69\nmm+++YZt27Yxf/589u/fz4YNGwgMDCQ+Ph43N7f7fi1C1Xfi2gle2/Ia7nbuLOi6ABdbl3I9fqWa\n7ng7Dydbvh8WhoOtksGLYzibnm/pkoS7aN26NatXrwZg5cqVtG3bFoDw8HB++eUXANPjtztz5gyN\nGjVizJgx9O7dmyNHjuDk5ERubm6J23fr1o358+ej1+sB7noqpmnTppw7d45Tp04BsGLFCjp06EBe\nXh7Z2dn06NGDmTNnmk75nD59mrCwMKZMmYK7uzsXLogP8oV/nc85z6ubX8VOacfCbgtxt3Mv9xoq\ndbADeNWwZ8UrYRhlmZcW7edKdqGlSxKAgoICvLy8TLcZM2Ywa9Ysli5dilqtZsWKFXz99dcAzJw5\nkxkzZhAaGsqVK1dwcblzdLNmzRr8/PwIDAwkMTGRyMhI3NzcaNOmDX5+fowfP/6W7YcNG4a3tzdq\ntZqAgAB++OGHUmtVqVQsXbqU/v374+/vj0KhYOTIkeTm5tKzZ0/UajUdOnTgq6++AmD8+PH4+/vj\n5+dH+/btCQgo9zesQgV1Nf8qIzaPQG/Us7DrQuo51rNIHVJ5TJa/XUhIiGzuhTaOXcrmxQX7qOVs\ny4+vRuDmaGvW/Vcm//zzD82aNbN0GfetoKAAOzs7JEli9erVrFq1it9++83SZQkPobL97pnTNc01\nhv45lJSCFBZ3W0wLd/OfBpUkKe5+WrZU+hH7DX71XFgc1YqL1woZsjSGHI3O0iUJ9ykuLs40jXHu\n3Ll8+eWXli5JEB5InjaP17a8xoXcC8x+YnaZhPqDqFLdHUMb1mT+Sy0Z/l0sw5bFsvzlUOxsrCxd\nlnAP7dq14/Dhw5YuQxAeikavYcz2MSRlJjGz00xa1Wll6ZKqzoj9hk5Na/HVgEAOJGcyamUcWr3R\n0iUJglBF6Yw6xu8cT2xKLB+3/ZgO9TtYuiSgCgY7QK+Aunzax5/tSWm8/WM8BmP5f44gCELVZpSN\nfPD3B+y4uIP3w97n6UZPW7okkyp1KuZmL4Z6k1Oo47M/EnFSWfNpHz/RVlQQBLOQZZlP93/KhjMb\nGBs8lgFNB1i6pFtU2WAHeLVDY7ILdczdcRoXO2smPtXU0iUJglAFzD40mzVJaxjaYiiv+L1i6XLu\nUCVPxdxsfHdfXgr3Zv7O08zdccrS5VQbV69eZeDAgTRq1IiWLVsSERHB2rVrH2mfH330EV988QUA\nkyZNYsuWLQ+1n/j4eDZu3PjAz3vxxRdRq9Wm+ewl1fWgYmNjGTNmTKmPnzt37pY5+Pfa/nYdO3bE\n3FOLq7tlx5ax8OhC+j3ej7davlUhzwRU6RE7FF+2PqW3H7kaPZ//mYSzypqXwhtYuqwqTZZlnn32\nWYYMGWIKpeTkZNavX3/Htnq93tTP5UFMmfLwfebi4+OJjY2lR48e9/2clJQU9u7dS3Jy8kMftyQh\nISGEhJQ+LflGsN/ooXOv7YWy9cuJX/gy7ku6+3Tng/APKmSoQzUYsQMoFBJf9A+gc9NafPDbMX6L\nv2Tpkqq0bdu2YWNjw8iRI03fa9CgAaNHjwZg2bJl9O/fn169etGtWzfy8vLo3LkzwcHB+Pv733Jx\n0ieffIKvry9dunQhKSnJ9P2bF8mIi4ujQ4cOtGzZku7du3PlSnE7544dOzJhwgRCQ0Np0qQJu3fv\nRqvVMmnSJNasWUNgYCBr1qy5pXaNRsPQoUPx9/cnKCiI7du3A8UtClJTUwkMDGT37t2lvvb4+HjC\nw8NRq9X06dOHa9eKG9QdOHAAtVpNREQE48ePN7UX3rFjBz179gRg586dBAYGEhgYSFBQELm5uUyc\nOJHdu3cTGBjIV199dcv2eXl5plrVarWpLUNpVq1aZbpidsKECUBxe+OoqCj8/Pzw9/c3vRuZNWuW\nqV3xCy+8cNf9Vhebzm1icvRk2tZry2dtP8NKUXGnUlf5EfsN1lYK5gwKZsiSGN758TCOtko6N6tt\n6bLK3h8TIeWoefdZxx+emlbqwwkJCQQHB991F9HR0Rw5coSaNWui1+tZu3Ytzs7OpKenEx4eTu/e\nvTl48CCrV6/m0KFD6PV6goODadmy5S370el0jB49mt9++w0PDw/WrFnD+++/z5IlS4DidwQxMTFs\n3LiRyZMns2XLFqZMmUJsbCzffPPNHXXNmTMHgKNHj5KYmEi3bt04ceIE69evp2fPnqZ+MaWJjIxk\n9uzZdOjQgUmTJjF58mRmzpzJ0KFDWbBgAa1bt2bixIklPveLL75gzpw5tGnThry8PFQqFdOmTeOL\nL77g999/B4r/ENwwdepUXFxcOHq0+L/vjT8iJbl8+TITJkwgLi6OGjVq0K1bN9atW0f9+vW5dOkS\nx44dAzC1IZ42bRpnz57F1tZWtCYG9lzaw8TdEwmqFcSMjjOwtrK2dEl3VS1G7DeorK1YNCSE5nWd\nGbXyIPvOZFi6pGrh9ddfJyAggFat/r1wo2vXrqZWurIs895776FWq+nSpQuXLl3i6tWr7N69mz59\n+mBvb4+zszO9e/e+Y99JSUkcO3aMrl27EhgYyMcff8zFixdNj/ft2xeAli1b3tKvvTR79uxh8ODB\nQHFzsAYNGnDixIn7ep3Z2dlkZWXRoUPxXOYb7X+zsrLIzc2ldevWQOmtidu0acPbb7/NrFmzyMrK\nuucpqi1btvD666+b/l2jRo1Stz1w4AAdO3bEw8MDpVLJoEGD2LVrF40aNeLMmTOMHj2aP//8E2fn\n4g6parWaQYMG8f333z/UqbKq5ODVg7y1/S0ec32M2Z1nY6e8e8voiqDa/RdzUlmzbGgoA76NZtjy\nWH4YHobay9XSZZWdu4ysy0qLFi1uOS0wZ84c0tPTbzk37ODgYPp65cqVpKWlERcXh7W1NT4+Pmg0\nGuDerX1lWaZFixZER0eX+LitbXHPICsrK1Onx3vtz9zud58TJ07k6aefZuPGjYSHh9/zw2FztCau\nUaMGhw8fZtOmTcyZM4cff/yRJUuWsGHDBnbt2sX69euZOnUqCQkJ1TLgEzMTeWPrG9RxqMP8LvNx\ntqkcrcGr1Yj9hpoONqx4JQxXe2uGLInh5NWSW78KD+eJJ55Ao9Ewb9480/cKCgpK3T47O5tatWph\nbW3N9u3bTR9Qtm/fnrVr11JYWEhubi7/+9//7niur68vaWlppmDX6XQkJCTctb67tftt3749K1eu\nBODEiROcP38eX1/fu7/g61xcXKhRo4bpHPyN9r81atTAycmJffv2AaW3Jj59+jT+/v5MmDCBkJAQ\nEhMT79ma+ObTSXc7FRMWFsbOnTtJT0/HYDCwatUqOnToQHp6OkajkX79+jF16lQOHjyI0WjkwoUL\ndOrUic8//5ysrCzy8vLu62dQlZzLPserm1/FwcaBBV0X4GZXeXruV8tgB6jjomLlsDCUVgpeWryf\nC5mlB4/wYCRJYt26dezcuZOGDRsSGhrKkCFDmD59eonbDxo0iNjYWEJCQli5ciVNmxZfbxAcHMyA\nAQMIDAykX79+tGvX7o7n2tjY8PPPPzNhwgQCAgIIDAxk7969d62vU6dOHD9+vMQPT0eNGoXBYMDf\n358BAwawbNky06j/fixfvpzx48ejVquJj49n0qRJACxevJgRI0YQERGBLMsltiaeOXOmaS1VOzs7\nnnrqKdRqNUqlkoCAgDumWf7nP//h2rVrpufc+KC3JJ6ennz22Wd06tSJgIAAgoODeeaZZ7h06RId\nO3YkMDCQqKgoPvvsMwwGAy+99JLpA+S33noLV9cq/K62BCn5KQzfPByAhV0X4unoaeGKHkyVadv7\nsJJScnn+22hc7a356dUIajmrLF3SI6vOrVMrqry8PNMi3dOmTePKlSumfvRVSVX43csozCDqzygy\nCjNY3H0xzdwqzuupdm17H5ZvHSeWDW1FWm4RgxfHkFWgtXRJQhV0Yxk9Pz8/du/ezX/+8x9LlySU\nIFeby2tbXiMlP4U5XeZUqFB/ENU+2AGCvGuwMDKEs+n5RC09QH7RvT9kE4QHMWDAAOLj4zl27Bgb\nNmzAw8PD0iUJtynUF/LG1jc4mXWSrzp9RVCtIEuX9NBEsF/X5jF3Zg8M4uilbEasiEWjE6vPC0J1\noTPoeHvH28SnxTOt3TTa1mtr6ZIeiQj2m3RvUYfP+6n5+1QGY1YdQm8QvdwFoaozGA28u+dd9lza\nw6TwSXT36W7pkh6ZCPbb9GvpxUe9mvPX8atM+OUoRtHLXRCqLKNsZHL0ZDad28S4kHH0a9LP0iWZ\nRfW74uA+RLVpSI5Gz4zNJ3BSKfmwV/MK2+xHEISHI8sy/z3wX9aeWsvIgJEMaTHE0iWZjdlG7JIk\nWUmSdEiSpN/NtU9LGv3EYwxr25Ble88xc8tJS5dT6dyY2vcoLl++zHPPPVfq41lZWcydO/e+t7/d\nzY3EzG38+PG0aNGC8ePHl8n+71diYiIRERHY2to+dGvhqmru4bl8/8/3vNTsJUYFjLJ0OWZlzhH7\nWOAfoHJcc3sPkiTx/tPNyNHo+HrrSZztrHmlbUNLl1Wt1K1b967BeyPYR40adV/bl6dvv/2WtLS0\n+7646WHbF99LzZo1mTVrFuvWrTP7viuzZceWMf/wfPo+3pf/a/V/Ve4duVlG7JIkeQFPA4vMsb+K\nQpIkPuurpod/Hab+fpwfYy9YuqRKLTk5mc6dO6NWq+ncuTPnz58Hii+lDw8Pp1WrVkyaNMk02j93\n7pypvW1CQgKhoaEEBgaiVqs5efIkEydO5PTp0wQGBjJ+/PhbtjcYDIwbN87U0nb27Nl3rW3r1q0E\nBQXh7+/Pyy+/TFFREVDcv+VG+9px48YB8NNPP5mu9mzfvv0d++rduzf5+fmEhYWxZs2aUl93VFQU\nb7/9Np06dTK10b2hXbt2t3SSbNOmDUeOHHngn3mtWrVo1aoV1tYVuxthefrpxE+mnuqTwidVuVAH\n843YZwL/BziVtoEkSSOAEQDe3t5mOmzZs1JIfDUgkLyiOCb+cgQHGyVPqyvP5cXTY6aTmJlo1n02\nrdmUCaET7r3hbd544w0iIyMZMmQIS5YsYcyYMaxbt46xY8cyduxYXnzxRebPn1/ic+fPn8/YsWMZ\nNGgQWq0Wg8HAtGnTOHbsmCkAb+7euGDBAs6ePcuhQ4dQKpVkZmaWWpdGoyEqKoqtW7fSpEkTIiMj\nmTdvHpGRkaxdu5bExEQkSTK1r50yZQqbNm2iXr16Jba0Xb9+PY6Ojqa6evXqVeLrhuJ+NFu2bMHK\n6tbe3sOGDWPZsmXMnDmTEydOUFRUhFqtZvv27bz11lt3HNPe3v6erRQE2HBmA1Ojp9Leq32F76n+\nKB55xC5JUk8gVZbluLttJ8vyAlmWQ2RZDqlsF2fYKq2Y/1IwLRvUYOzqQ2xPTLV0SZVSdHS0qWXt\n4MGD2bNnj+n7/fv3B0pvaRsREcGnn37K9OnTSU5Oxs7u7q1Tt2zZwsiRI02nN260CC5JUlISDRs2\npEmTJsC/7XadnZ1RqVQMGzaMX3/9FXt7e6B49BwVFcXChQsxGO59vUNprxugf//+d4T6je///vvv\n6HQ6lixZQlRUFFDc5yY+Pv6Omwj1e9t+fjvv73mfkDohfNnhywrfU/1RmGPE3gboLUlSD0AFOEuS\n9L0syy+ZYd8Vhr2NksVRrRi0cD8jv49j2dBQIhpX/G5vDzOyLi8P8hZ44MCBhIWFsWHDBrp3786i\nRYto1KhRqdubo6WtUqkkJiaGrVu3snr1ar755hu2bdvG/Pnz2b9/v6lNQHx8PG5u9/+7cHNdN7cv\nvpm9vT1du3blt99+48cffzStW3qvEfucOXNYuHAhABs3bqRu3br3XVdVtu/KPsbtHEdzt+bMfmI2\nKmXl7wl1N488Ypdl+V1Zlr1kWfYBXgC2VbVQv8FZZc3yl0PxrmnPsOUHOHS+9Dapwp1at25talm7\ncuVK2rYtvrovPDzc1L+9tJa2Z86coVGjRowZM4bevXtz5MiRe7a0nT9/vqkH+91OxTRt2pRz585x\n6lTxYuc32u3m5eWRnZ1Njx49mDlzpunUyunTpwkLC2PKlCm4u7tz4cLdP3sp7XXfy7BhwxgzZgyt\nWrUyveO414j99ddfN31PhHqx+NR4xmwbQwOXBszrMg8H65L/mFYl4gKlB1TTwYaVw8Jwd7JlyJIY\njl/OsXRJFVJBQQFeXl6m24wZM5g1axZLly5FrVazYsUKU3fDmTNnMmPGDEJDQ7ly5UqJLW3XrFmD\nn58fgYGBJCYmEhkZiZubG23atMHPz++OaYXDhg3D29sbtVpNQECAaVHtkqhUKpYuXUr//v3x9/dH\noVAwcuRIcnNz6dmzJ2q1mg4dOpja5o4fP960dmj79u0JCAi468+itNd9Ly1btsTZ2ZmhQ4fe1/Yl\nSUlJMf38P/74Y7y8vMjJqT6/s4mZiYzaOgoPOw8WdF2Ai+2dv1tVUbVv2/uwLl4roP/8aLR6Iz+O\njKCxx6PP2zaXytY6taCgADs7OyRJYvXq1axateqWBa2rq8uXL9OxY0cSExNRKCrHGKwi/e6dzT5L\n1J9R2FjZsPzJ5dR1rPzvYETb3jLmVcOelcPCkCR4aZFYqONRxMXFmaYxzp07ly+//NLSJVncd999\nR1hYGJ988kmlCfWK5FLeJYb/9e9CGVUh1B+EGLE/on+u5PDCgn242Fnz08gIaleAhToq0qhJqF4q\nwu9eWkEaQ/4cQlZRFku7L8W35v0tbVgZiBF7OWnm6czyl0PJyCvipUX7ycwXC3UIgqVkabIYsXkE\n6YXpzO8yv0qF+oMQwW4GgfVdWRzVivOZBUQu2U+ORmfpkgSh2snT5jFyy0jO55znmye+Qe2htnRJ\nFiOC3UzCG7kxf3BLklJyeXnpAQq0YhUmQSgvhfpC3tj2BkmZSczoOINQz1BLl2RRItjNqJNvLWa9\nEMTB89cY8V2cWIVJEMqBzqDjrR1vcfDqQT5r9xkd6newdEkWJ4LdzJ7y9+Tz5wLYcyqdN344hK6a\nrsJ09epVBg4cSKNGjWjZsiURERGsXbv2kfb50UcfmVrPTpo0iS1btjzUfuLj49m4ceMDP+/FF19E\nrVab5rOXVJe5zZo1i2bNmjFo0KAy2f/9ysjIoFOnTjg6OvLGG29YtJab6Y16JuyewN+X/ubDiA95\nsuGTli6pQhALbZSB51p6UajV88FvCbzz42G+GhCIlaLqdZArjSzLPPvsswwZMsR0YVBycjLr16+/\nY9uHbVc7ZcqUh64vPj6e2NhYevTocd/PSUlJYe/evSQnJz/0cR/G3Llz+eOPP2jY8P5aRpdV+1+V\nSsXUqVM5duwYx44dM/v+H4ZRNvLR3o/YnLyZ8SHjq8zqR+YgRuxlZHCEDxOebMr6w5d5f+3RUvuR\nVEXbtm3DxsaGkSNHmr7XoEEDRo8eDcCyZcvo378/vXr1olu3buTl5dG5c2eCg4Px9/e/5eKkTz75\nBF9fX7p06UJSUpLp+zcvkhEXF0eHDh1o2bIl3bt358qVKwB07NiRCRMmEBoaSpMmTdi9ezdarZZJ\nkyaxZs0aAgMDWbNmzS21azQahg4dir+/P0FBQWzfvh0oblGQmppKYGAgu3fvLvW1x8fHEx4ejlqt\npk+fPly7Vtx2YtasWab2vy+88AIAO3fuJDAwkMDAQIKCgu5ojzBy5EjOnDlD7969+eqrr8jMzOTZ\nZ59FrVYTHh5uauP70UcfMWLECLp160ZkZOQt+xg8ePAtP89BgwaV+Af2XhwcHGjbti0qleWn80Lx\n4GF6zHR+O/0bowJGEdki8t5PqkbEiL0MvdaxMflFer7ZfgoHWyX/ebpZufd+Tvn0U4r+MW/bXttm\nTanz3nulPp6QkEBwcPBd9xEdHc2RI0eoWbMmer2etWvX4uzsTHp6OuHh4fTu3ZuDBw+yevVqDh06\nhF6vJzg4mJYtW96yH51Ox+jRo/ntt9/w8PBgzZo1vP/++yxZsgQoHsHGxMSwceNGJk+ezJYtW5gy\nZQqxsbF88803d9Q1Z84cAI4ePUpiYiLdunXjxIkTrF+/np49e97SI70kkZGRzJ49mw4dOjBp0iQm\nT57MzJkzmTZtGmfPnsXW1tbU6veLL75gzpw5tGnThry8vDtCc/78+fz5559s374dd3d3Ro8eTVBQ\nEOvWrWPbtm1ERkaa6omLi2PPnj13dL0cNmwYX331Fc888wzZ2dns3buX5cuXk5SUxIABA0p8DTt2\n7MDV1fWur9PSZh+azQ+JPxDZPJKRASPv/YRqRgR7GXunWxPyivQs3nMWB1slb3dtYumSyt3rr7/O\nnj17sLGx4cCBAwB07drV1NhKlmXee+89du3ahUKh4NKlS1y9epXdu3fTp08fU7vc3r1737HvpKQk\njh07RteuXYHiBTY8Pf/tl9+3b1+guO/Kzf3aS7Nnzx7TO4umTZvSoEEDTpw4gbPzvRcGy87OJisr\niw4dij+8GzJkiKkdsVqtZtCgQTz77LM8++yzQHH737fffptBgwbRt29fvLy87lnbjWZpTzzxBBkZ\nGWRnZwPFP5uSWhl36NCB119/ndTUVH799Vf69euHUqnE19f3nn+kKqoFRxaw8OhC+j3ej3Eh46rk\nQhmPSgR7GZMkiUk9m1Og1TNr60kcba0Y0b5xuR3/biPrstKiRQtTAEHxKDg9PZ2QkH8vmLu5Xe3K\nlStJS0sjLi4Oa2trfHx80Gg0wL1b+8qyTIsWLYiOji7x8RtL01lZWZk6Pd5rf2Vhw4YN7Nq1i/Xr\n1zN16lQSEhKYOHEiTz/9NBs3biQ8PJwtW7bQtGnTB6rtxs+ntPa/UHw6ZuXKlaxevdr0TuZeI/bt\n27czefJkABYtWnTLfztLWp6wnNmHZtOrUS8mRVTN1Y/MQZxjLwcKRfESez3Vnny6MZHv95XvB3Dl\n7YknnkCj0TBv3jzT9woKSu+lk52dTa1atbC2tmb79u2mDyjbt2/P2rVrKSwsJDc3l//97393PNfX\n15e0tDRTsOt0OhISEu5a393a/bZv356VK1cCxasbnT9/Hl/f+7t60cXFhRo1apjOwd9o/2s0Grlw\n4QKdOnXi888/Jysri7y8PE6fPo2/vz8TJkwgJCSExMS7nzK7ubYdO3bg7u5+X+8koqKimDlzJlD8\nRxcwjdhLurm6utKnTx/TvytKqK9JXMMXsV/QtUFXprSZgkIS8VUaMWIvJzeW2NPoDHzw2zEcbK3o\nE3T3t94cI0MvAAAgAElEQVSVlSRJrFu3jrfeeovPP/8cDw8PHBwcmD59eonbDxo0iF69ehESEkJg\nYKBp1BocHMyAAQMIDAykQYMGtGvX7o7n2tjY8PPPPzNmzBiys7PR6/W8+eabpgArSadOnZg2bRqB\ngYG8++67t4xcR40axciRI/H390epVLJs2bL7XpAaYPny5YwcOZKCggIaNWrE0qVLMRgMvPTSS2Rn\nZyPLMm+99Raurq588MEHbN++HSsrK5o3b85TTz11131/9NFHDB06FLVajb29PcuXL7+vmmrXrk2z\nZs1Mp4Aelo+PDzk5OWi1WtatW8dff/1F8+bNH2mf9+u3U7/x8f6P6eDVgentpqNUiOi6G9EErJxp\ndAZeWX6AfWcymTMwmCf96pj9GBWhEZNQcRQUFODv78/BgwdL7HVvTmXxu/fn2T+ZsHsCoXVC+abz\nN9ha3f8f2qpGNAGroFTWViwYHEJgfVdGrzrI9iSxfqpQdm6ctx89enSZh3pZ2HZ+G+/ufpdAj0C+\n7vR1tQ71ByGC3QIcbJUsiWqFbx0nRq6I4+9T6ZYuSaiiunTpwvnz53nzzTctXcoD+/vS34zbOY5m\nbs2Y03kO9tb2li6p0hDBbiEudtaseDmMhu4ODFseS8zZ0tfkfBjV6YIooWIw5+/cgZQDjN0+lsau\njZnXZR6ONhVnhbLKQAS7BdVwsGHFK2HUdVUxdGkMB820OLZKpSIjI0OEu1BuZFkmIyPDLFemxqfG\n8/rW16nnWI9vu35bbdYpNSfx4WkFcDVHw/PfRpOZr2XV8HD86j3aL7JOp+PixYumueCCUB5UKhVe\nXl5YW1s/9D6OZxxn2KZh1FDVYNmTy/Cw9zBjhZXf/X54KoK9griUVcjz86PJ1+pZPSKcpnXuPT9Z\nEKqSk9dO8vKml7FT2rH8yeV4Onre+0nVjJgVU8nUc7Vj1fBwVEorBi3cz6nUki+gEYSq6Fz2OYb/\nNRxrhTWLuy0Wof6IRLBXIN5u9vwwPAxJkhi4cD/n0vMtXZIglLmLuRcZ9tcwZGQWdVtEfef6li6p\n0hPBXsE08nDkh+Fh6I0ygxbt5+K10i/FF4TKLiU/hWF/DaNAX8CCrgto5NrI0iVVCY8c7JIkqSRJ\nipEk6bAkSQmSJE02R2HVWZPaTqx4JZRcjY6BC/dzJbvQ0iUJgtmlF6Yz/K/hZBVl8W2Xb/GteX89\neYR7M8eIvQh4QpblACAQeFKSpHAz7Ldaa1HXhRWvhJGZr2XQwv2k5ooZLkLVkanJZPhfw7lacJU5\nnefg7+Fv6ZKqlEcOdrlY3vV/Wl+/iQnUZhBQ35VlQ1uRkqNh0ML9ZOQVWbokQXhkWZoshv81nAu5\nF5j9xGxa1m557ycJD8Qs59glSbKSJCkeSAU2y7K8v4RtRkiSFCtJUmxaWpo5DlsthPjUZPGQVpzP\nLGDw4hiyCrSWLkkQHlp2UTYjNo/gXPY5ZnWaRZhnmKVLqpLMEuyyLBtkWQ4EvIBQSZL8SthmgSzL\nIbIsh3h4iIsOHkREYzcWRIZwKjWPIUtiyNHoLF2SIDywHG0Or25+lVNZp5jZaSat67W2dElVllln\nxciynAXsAJ40534F6NDEg7mDgkm4nMPQpQfIL7r3akCCUFHkafN4bfNrJF1LYkbHGbTzurO3vmA+\n5pgV4yFJkuv1r+2ALoB5V08WAOjSvDazXwwi/kIWryw/QKHWYOmSBOGeCnQFjNo6ioSMBL5o/wUd\n63e0dElVnjlG7J7AdkmSjgAHKD7H/rsZ9iuU4Cl/T2Y8H8D+s5kM/y4WjU6Eu1Bx3Qj1I2lHmN5+\nOp0bdLZ0SdXCI68vJcvyESDIDLUI9+mZwHroDDLjfz7M8O9iWRgZgsraytJlCcItCvWFjNk2hkOp\nh/is7Wd09+lu6ZKqDXHlaSX1XEsvPu+nZs+pdEasiBMjd6FCKTIUMXbbWGJSYvi4zcf0aNTD0iVV\nKyLYK7H+IfWZ3lfNrhNpvCrCXaggtAYtb25/k+gr0UxuPZlejXtZuqRqRwR7Jfd8q/pM6+vPzhNp\nvPZ9HEV6Ee6C5egMOt7Z8Q57Lu3hw4gP6fN4H0uXVC2JYK8CXgj15rO+/mxPSuO17w+KcBcsQmfU\nMX7XeHZc3MH7Ye/zXJPnLF1StSWCvYp4MdSbT/v4sy0xlddXinAXypfeqGfirolsPb+ViaETeaHp\nC5YuqVoTwV6FDAzz5uNn/djyTyqvrzyEVm+0dElCNaAz6piwawJ/Jf/FuJBxDGo2yNIlVXsi2KuY\nl8IbMPWZFmz55yqv/3BQhLtQpm4P9SEthli6JAER7FXS4AgfJvduwebjVxm96iA6gwh3wfx0Bh3j\nd45nc/JmxoeMF6FegYhgr6KGtPbhw17N2ZRwlTGrDolwF8xKZ9Axbuc4tp7fyv+1+j8iW0RauiTh\nJiLYq7ChbRryQc/m/HEshbGrRbgL5qEz6Hhn5ztsu7CNiaETGdx8sKVLEm7zyC0FhIrtlbYNkWWZ\njzf8Axzi6xeCsLYSf8+Fh6M1aHlnxzvsuLiDd0PfZWCzgZYuSSiBCPZqYFi74gWCP97wD3rDQb4Z\nGIyNUoS78GC0Bi1v73ibnRd38l7Ye7zY9EVLlySUQvzfXU0Ma9eIyb1b8Nfxq4z8XrQfEB5MkaGI\nN7e/yc6LO/lP2H9EqFdwItirkSGtffikjx/bElNF4zDhvt0I9d2XdvNB+AcMaDrA0iUJ9yCCvZoZ\nFNaAz/up2X0yjZeXHaBAK1ZiEkpXZChi7Pax7Lm0h0kRk3je93lLlyTcBxHs1dDzrerzZf8A9p3J\nIGrpAfLEMntCCW70U//70t98FPER/Zv0t3RJwn0SwV5N9Q324qsBgcQlX2PIkhhyxQLZwk3ydfmM\n2jKK6MvRTGk9hX5N+lm6JOEBiGCvxp4JrMfsF4M4fCGLlxbHkF0owl2AHG0OIzaP4FDqIaa1myZa\n71ZCItiruR7+nswdFMzxy9m8tGg/WQVaS5ckWFCWJothm4ZxPOM4X3T4Qqx8VEmJYBfo1qIOCwaH\nkHQ1lxcX7iczX4R7dZRemM7QTUM5nXWarzt9TZcGXSxdkvCQRLALAHRqWotFkSGcScvjhQXRpOZo\nLF2SUI5S8lMY+udQLuVdYk6XObT3am/pkoRHIIJdMGnfxIOlQ1tx8Voh/b+N5kJmgaVLEsrBpbxL\nRP0ZRVphGvO7zCfcM9zSJQmPSAS7cIvWjd35flgY1/K1PP9tNKfT8ixdklCGknOSGfLHEHK1uSzq\ntojg2sGWLkkwAxHswh2CvWuw5tUIdAYjz8+PJuFytqVLEsrA6azTRP0ZhdagZUn3Jfi5+1m6JMFM\nHjnYJUmqL0nSdkmS/pEkKUGSpLHmKEywrGaezvz4agS2SgUvLNhHXHKmpUsSzCghPYGoP6OQkFj6\n5FJ8a/pauiTBjMwxYtcD78iy3AwIB16XJKm5GfYrWFgjD0d+eq01bg42vLQohj0n0y1dkmAGMVdi\neHnTyzhYO7DsyWU0dm1s6ZIEM3vkYJdl+Yosywevf50L/APUe9T9ChVDPVc7fhwZQQM3e15edoC/\nElIsXZLwCLad38ZrW17D08GT5U8ux9vZ29IlCWXArOfYJUnyAYKA/SU8NkKSpFhJkmLT0tLMeVih\njNVyUrF6RDjN6zrz2sqDrDt0ydIlCQ9h/en1vL3jbXxr+rLsyWXUdqht6ZKEMmK2YJckyRH4BXhT\nluWc2x+XZXmBLMshsiyHeHh4mOuwQjlxtbfh+2FhhPrU5K0f4/l+X7KlSxIewPfHv+f9Pe8TUieE\nRd0W4apytXRJQhkyS7BLkmRNcaivlGX5V3PsU6h4HG2VLB3aiid8a/GfdceYvfUksixbuizhLmRZ\nZm78XKYfmE4X7y7M7TwXe2t7S5cllDFzzIqRgMXAP7Isz3j0koSKTGVtxfzBLekbVI8vN5/gw/UJ\nGIwi3Csio2xkWsw05h2ex7OPPct/O/wXGysbS5cllANzrHnaBhgMHJUkKf76996TZXmjGfYtVEDW\nVgq+6B+Au5MtC3adISNPy4wBAdgqrSxdmnCdzqhj0t+T+P3M7wxpPoR3Qt6heAwmVAePHOyyLO8B\nxG9MNaNQSLzXoxnujjZ8ujGRawVavh3cEieVtaVLq/bydfm8s+Md/r78N2OCxjDMf5gI9WpGXHkq\nPJIR7Rsz4/kAYs5mMuDbfaTmiuZhlpRemM7Lm15m35V9TG49meHq4SLUqyER7MIj6xvsxcIhIZxN\nz+e5edEkZ+RbuqRqKTknmcEbB3M2+yyznphF38f7WrokwUJEsAtm0cm3Fj8MDyNXo6PfvL0cuyT6\ny5Sno2lHGbxxMPm6fBZ3Wyza7lZzItgFswnyrsFPI1tjY6VgwLfR7EhKtXRJ1cKui7t45a9XsLe2\nZ0WPFfh7+Fu6JMHCRLALZvVYLUd+HdUGbzcHXlkeyw/7z1u6pCrt15O/MmbbGHycffi+x/c0cG5g\n6ZKECkAEu2B2dVxU/DQygraPufPe2qNM/zMRo5jrblayLDMvfh4f7v2QcM9wlj65FHc7d0uXJVQQ\nItiFMuFoq2TxkBAGhnkzb8dpxqw+hEZnsHRZVYLWoOW9Pe8x9/BcejfuzezOs3GwdrB0WUIFYo4L\nlAShREorBZ8864d3TXum/ZFISraGhZEh1HAQVz8+rGuaa7y5/U0Oph5kdNBohvuL6YzCncSIXShT\nkiQxskNjvhkYxJFL2fSdt5dz6WI65MM4k32GQRsHcSz9GP9t/19GqEeIUBdKJIJdKBc91XX5YVgY\nWQVa+s7bS8xZsSLTg9h/ZT8vbXypeDpj98U82fBJS5ckVGAi2IVyE+JTk19HtcHVzppBi/ax5oCY\nMXM/1p5cy8jNI6llV4uVPVYSWCvQ0iUJFZwIdqFcNXR3YO2oNoQ3cmPCL0eZ/L8E9AajpcuqkAxG\nAzPiZjBp7yRa1WnFih4r8HLysnRZQiUggl0ody721iyNasXLbRqy9O9zDF12gOwCnaXLqlBytDm8\nse0Nlh5byvNNnmdOlzk42ThZuiyhkhDBLliE0krBpF7N+byfmn1nMugz929Op+VZuqwK4UzWGQZu\nGMi+y/v4IPwDPoj4AGuF6Jop3D8R7IJFPd+qPj8MDye7UMezc/5m54nqvR7ujgs7GLhxILnaXBZ3\nX8zzvs9buiShEhLBLlhcK5+a/PZGG+q52jF0aQxztp+qdleqyrLMgiMLGLNtDN5O3qzpuYbg2sGW\nLkuopESwCxWCVw17fh3VmqfVdfnvpiRGrIgju7B6nHfP1+Xzzs53mH1oNj0a9eC7p76jjkMdS5cl\nVGIi2IUKw95GyawXAvmwV3N2JKXyzDd7SEzJsXRZZerUtVO8uOFFtp7fyjst3+Gztp+hUqosXZZQ\nyYlgFyoUSZIY2qYhq0aEk6810GfOXn6Lv2TpssrE72d+Z+DGgeQU5bCo2yKi/KLElaSCWYhgFyqk\nVj412TC6Lf71XBi7Op6P1ieg1VeN+e5FhiKmRk/l3d3v0tytOT/1+olWdVpZuiyhChHBLlRYtZxV\nrBwexittG7Js7zn6z9/L+YwCS5f1SC7lXSLyj0h+PPEjQ/2GsqjbIjzsPSxdllDFiGAXKjRrKwUf\n9GzO/JeCOZuez9OzdvO/w5ctXdZD2ZK8hef/9zwXci7wdaevebvl2ygVosGqYH4i2IVK4Uk/TzaO\nbcfjtR0ZveoQE385QqG2cvR3L9AV8NHej3hrx1vUd6rPmp5reML7CUuXJVRhItiFSsOrhj1rXo3g\n9U6NWRN7gV6VYNZMYmYiL2x4gV9P/srLfi+z4qkV1Heub+myhCrOLMEuSdISSZJSJUk6Zo79CUJp\nrK0UjO/elBUvh5FVoOOZb/5m+d5zFe6CJlmWWXF8BQM3DCRPm8eCbgt4q+VbWFuJ1gBC2TPXiH0Z\nIBpEC+Wm7ePu/DG2HRGN3fhwfQKRS2K4nFVo6bIASC1I5bWtr/H5gc9pU68Nv/T+hXDPcEuXJVQj\nkiybZ6QjSZIP8Lssy3732jYkJESOjY01y3GF6k2WZVbFXODjDcexUkhM7t2CPkH1LDIfXJZlfj/z\nO5/FfIbOoOOdkHcY4DtAzE2vJGSjEQyGm+5lMBqQDQYwGk33t25jvP6Y8fq2xpKfYzQW7082omrR\nAmXNmg9VoyRJcbIsh9xru3L7SF6SpBHACABvb+/yOqxQxUmSxMAwb9o85sa4nw7z9o+H+SvhKp/0\n8cPN0bbc6kgvTGdq9FS2XdhGgEcAn7T9hAbODcrt+JWZbDRiLCjAmJ+PMb8AuUiDUaNBLioqvtcU\nXf/eTfcaDcai4seMRRrkIi2yXoes04FOj6zTIetvvy/l8etfYyifD+PrL1yAY7t2ZXoMMWIXqgyD\nUWbR7jN8+dcJnO2UTHnGj6f86pT5iHnTuU18su8T8nX5vBH0BpHNI7FSWJXpMSsKWZaRCwowZGf/\ne8vKwpB1/eucbIx5+ddD+6ZbXh6GgutBXvBw1yZItrZIKhUKW1skGxska+vim1Jp+hrr618rb3+s\n+B7lTY8rlUhKK1BYIVkpQFKAlQJJYVV8b2UFihv317dRWCEpJChxm9vuJQlJocCmUSOsnJ0f7jVX\ntBG7IJQ1K4XEqx0a08HXg3E/HWbUyoN0aVabqc+2wNPFzuzHSy1IZVrMNDYnb6aFWws+afsJjV0b\nm/045U3W6dBnZqJPT8eQno4+PR19Wjr6jAz06WkY0tLRZ13DkJ2NMSu7eBRcCsnWFoWDAwpHx+J7\nB3us3N2waeBd/G/7mx9zQGFvj6SyRaFSFT9XpUKyVaFQ3RTiKlVxkCvEpL7SiBG7UCXpDUaW/n2O\nLzcnoVQomPBUUwaFeqNQPPro3Sgb+THpR2YenIneqGdkwEiiWkRViouNZKMRQ0YGusuX0V25gu7y\nleL7K5fRX//acO1aic9VODmhdHdH6eaGlZsbVi4uxTfXG/euWLm4oHBxwcrFFSsXZxQq0dDMnO53\nxG6WYJckaRXQEXAHrgIfyrK8uLTtRbAL5eV8RgHvrT3KnlPphDSowWd9/Xm89sMvMXfi2gkmR0/m\nSNoRwj3D+SD8A7ydK9ZnRrJOh+7yZbTJyWjPJRffnz+P9nwy+stX7hhhK+ztsa5XF6WnJ9Z1PFHW\nqlUc4B7XQ9zdA6W7mwjpCqBcg/1BiWAXypMsy/x68BJTNxwnT6NnaBsfxnR+HCfV/c8pL9AV8O2R\nb/ku4TucbJwY32o8PRv1tOiMF2NBAUWnz1B08iRFp05RdOok2uRkdJcug15v2k5hb4+1TwNsvBtg\n41WvOMDr1i2+eXqicHISM3cqCRHsgnCbjLwi/rspiTWxF3B3tOXdp5rec2qkUTay4cwGvor7irTC\nNJ5p/AzjQsbhqnItt7plgwHt2bNoEpOKQ/x6kOsuXIDr//9KNjbYNGyIjY8PNg0aFN98iu+t3NxE\ncFcRItgFoRSHL2QxaX0Chy9kEdKgBh/1boFfPZc7tjuWfozPYj7jSNoR/Nz8mBA6gcBagWVam2ww\noD1zhsKEBDQJx9EkJKBJTPx35oiVFTY+Ptg+/ji2jz+G7WOPY/v449h410dSVvxz/MKjEcEuCHdh\nNMr8fPAi0/9IJLNAS79gL97u2oS6rnakFqQy6+Asfjv9G24qN8YGj+WZx55BIZl/FoYuJYXCQ4co\njI+n8MjR4hAvLL6CVrKzQ9W0KaoWLYpvzZth07AhChsbs9chVA4i2AXhPmQX6vhm20mW701Gsiok\n0P8Qp4v+xCDrGdxsMCPUI3C0cTTLsWSdDk1i0vUgP0TBoXj0V64AxdMCTQHeojl2LVpg06hR8fxn\nQbhOzGMXhPvgYmfNO90b4VBrN8uPL+V4QT7kBRHZ9FVGBYSjsn74YDVqtRTGx1Owbx8FB2IpPHoU\nWaMBQOnpiX1QIHZDh2IXFISqqW/xBTOCYAYi2IVqS2vQsu7UOr498i2pBam0rdeWXvVfYfXfRuZs\nTuOX/TsY1akxz4fUv6+Alw0GNP8kUrAvmvzofRTExRUHuUKBqnlzXJ/vj31QEHaBgVh7epbDKxSq\nK3EqRqh2NHoNv5z8hSXHlpBakIraQ82bwW/esu7o3lPpfLXlBAfOXaO2sy2vdWjMC6HetwS8LMto\nz50jPzqaguh95MfEYMzOBsDmscY4hEfg0DoC+1atsHJ6+LnzgnCDOMcuCLcp0BXwY9KPLEtYRoYm\ng+Bawbwa8CoRnhElTgeUZZno0xnM3HqSmLOZ1HKy5Q21C92LLmKIjSF/3z70KSkAKOt6Fgd5RDj2\nYWFY16pV3i9PqAbEOXZBuO5q/lVWJa7ipxM/kaPNIcwzjP+q/3vLCL0kkiQR5mHNokZ5nDm5l5wN\n0XisuEImUOTghGN4OHVGjsQhIhxrb28xV1yoMESwC1VWQkYCK46vYNPZTRgx0tm7M5HNI+86F92o\n0VAQF0fBvn3kR+9Dc/w4GI2o7OyoGRJCVtN+/CJ78n26NZLCit5SXV6xdqWFCHWhAhHBLlQpRYYi\nNidv5qeknziYehB7pT0vNH2BQc0G4eXkdcf2sl6P5tgx8q8HeeGhQ8haLSiV2AUE4P7aazhEhGOn\nViPZ2OANqIFhmQUs+fssaw5c4NdDl2jZoAaDwrzp4e/5SDNpBMEcxDl2oUo4nXWan0/8zP/O/I/s\nomzqO9VngO8A+j7eFyebfz+4lGWZopMnTSPyggMHMOblAWDbtCkO4eHF58lDQlA4ONzzuNkFOn6K\nu8AP+89zJj0fV3trngv24sUwbxp7mGf+uyDcID48Faq8HG0Om89tZv3p9RxMPYhSoaSzd2eea/Ic\noXVCUUgKZFlGd+EC+fv3U7BvP/kx+zGkpQNg7e39b5CHhT30cmVw/YPWMxms3H+eTcdS0BtlWvnU\n4NmgevT0r4uLvZijLjw6EexClVRkKGLnhZ1sPLuRXRd3oTPq8HH2oe/jfenduDdudm7oUlLI37eP\ngv0x5O/fh/5y8dWdVh7uOISG4dA6AofwcKzr1SuTGlNzNfwcd5FfD17iVGoeNlYKnmhai2eD6tGp\nqQe2SnGqRng4ItiFKqNQX0j05Wi2nt/KtvPbyNPl4W7nzpM+T/J0o6fxpQ6FMTHk74+hYN8+tMnJ\nAFi5uGAfFoZ9eBgOYWHFl+iX44ecsiyTcDmHXw9eYv3hy6TnFeGsUtK5WW26t6hDhyYe2NmIkBfu\nnwh2oVLL0mSx8+JOtp3fxt7Le9EYNDjZONHJqyO9bFvR5LwBzaFDFMYdRHvuHAAKR0fsQ0KKgzw8\nHNsmTSrM8ml6g5G/T2ewPv4yWxOvklWgQ2WtoGOTWnT3q00n31q42ovmXsLdiWAXKhW9Uc+x9GNE\nX4lm3+V9xKfFY5SNeNrW4lk5gPA0F9xPpqE5GI8hIwMoHpHbBQdjFxyEQ1gYqubNK0XrWr3BSMzZ\nTP5MSGFTQgpXc4pQSKD2cqVDEw/aN/EgsL4rVmZYxk+oWkSwCxWaLMuczTlLbEosey/vJeZKDLna\nHDyvSbTPrUura67Uv6RFceKcqXGWtZcX9i2DsQtuiX3L4OJTKxVkRP6wjEaZwxez2JGUxq6TaRy+\nkIVRLm5O1vYxd8Ib1SS0oRuP13I0y3qtQuUmgl2oUHQGHQkZCRxKPVR8u3oQ69QsfFJlAtIdUKfb\n45Gci1Ve8YISkkqFqnlz7Pz9sAsMxC44GOvatS38KspeVoGWPafS2ZmUxu6T6aTkFP9Rc7W3JqRB\nTUIb1iC0oRvNPZ2xUVbuP2rCgxPBLliMzqjjTNYZjmcc55/Mfzh16SiFSYl4pmjxTpVpkmmL11U9\n1prr63JaWWHbpAl2/v6o/P2w8/fH9rHHKsVplbIkyzIXMguJOZdJzNkMDpy7xtn0fABsrBQ083TC\n38sFtZcrAV6uPFbLUZy+qeJEsAtlTpZlMjQZnMk6w+msU1xIPkZm0lH0585TK0NH3QyonwG1sv79\nHZMcHVE19UXVxBdbX19Uvk2wbdoUhUplwVdSeaTmaDhw7hpHLmZx5GI2Ry9lk1dU/AfSztqKZp5O\n+NZxokltJ3xrO9GkjhPujrYWrlowFxHsgtkU6gu5nHeZi1nnSUlOIP3McfIvnEO+nIJrmoa6mTJ1\nM8G+6N/nGG2UWHnXx/ExX1RNfbFtUhziyrp1RbMsMzIaZc5m5HPkYhaHL2STmJJDUkou1wp0pm1q\nOtjQpLYjjTwc8XGzp4GbAz5uDnjXtBfTLSsZ0d1RuC+yLJOnyyM17yppKae5dukMWZfOkp9yEX3K\nVaxTr+GUoaFWtoxbDtQy3vRcCXRuzkgNvHBs54vL482wbdgI24Y+KD09K/0Hm5WBQiHR2MORxh6O\n9Akq7oUjyzLpeVpOXM0lKSWXE1eLb38cvXJL4APUcVbRwM2e+jXtqeuiwtPVDk8XFXVd7ajjosJZ\nJa6YrYxEsFdBsixTqC8kM/cq2VcvkJN2mfz0KxRmpFKUmYY2MwMyrqG8loddViGuuUZc88DFCC63\n7avAVYWudh2sGtZBqu+Dk08TXH0ex8bLC6Wnp1hYuQKSJAkPJ1s8nGxp85j7LY9lF+hIzsznXEYB\nyenF9+cy8tl9Mo3U3CJufwPvaKvE00VFHRcV7o62uDnY4O50/d7RFjfH4vuaDjai+VkFYpZglyTp\nSeBrwApYJMvyNHPst7qSZZlCbT55uZnkZ6WRn5VGYXYGmuxrFOVkosvNQZ+bgzE3Dzk/Hzm/EEV+\nIYoCDTZ5Rdjl63EslLHTggJwvX67WaG9FYWudhhq1EZ+zI28WrWw86yLs6cP7vWboKpTF6WHOwpb\ncX62KnGxt0Zt74ra6/bfCNAZjFzN0ZCSreFytoYrWYVcydZwOauQqzkazqTlk55XRJHeWMKei/8I\nuNhZ42xnjYudEmeV9U3/vvF18TYONkocbJXY21hhb6PE3tYKe2srlFbiXZ45PHKwS5JkBcwBugIX\ngUmObo0AAA0hSURBVAOSJK2XZfn4o+67rBmNRgwGHUa9Hr1Bi9Ggx6DXIRv06HVa9DotOm0huqJC\n9FoNem0h+qIi9FoNBq0Gg1aLQVuEUVeEQatF1mox6LTIOi1GrRajToes02HUFoGmCFlThFSkhSIt\niiIdVlo9Cq0epdZQfNPJ2OiM2OjA2vBvnVaA4/Xb7YpsJDQqBTqVEoOdLXo3N4oaOaJzdSHPtSY2\nNd2xd6+No4cnzh5eOHvUw7pGDSQx0hZuY22lwKuGPV417EvdRpZlCrQGMvK0pOcXkZGnJSOviPS8\nIjLytWQX6sgp1JNTqCM5o4AcjY7sQh0FWkOp+7yZjVKBw42wt7HC3laJg40VKmsrbJUKbJSKm+6t\nbvn6xmO3P25tpcBKIWFtJV2/v/nfCpQKCeWNxxQKrKyu31/fpjJ+JmSOEXsocEqW5TMAkiStBp4B\nzB7sf34yEuXOA0hG+dabfOMeJKOM4vrXCqOMwsi/X8sgGUEh/3t7EArA5vrtQRj+v727j63rvus4\n/v6ecx/9ENux4zw5Tt0ubZ22a7dmyaaqD2KwdWspMIHEJhCCP8IfTNpEKzTYP/wNEuMPkKBCSEhM\nQiCokGCoDIllHdC1aWlpQtq15NFu85wmdvxwn3788T3OdZw4dux774mvPy/pp/Nwj+/9nkT55Nzf\n+fn8DEpZKOciytmIai6mkoup5jLUOovMbswyk89BPgeFAlbMY4UC2WInme4N5Lp7KPRspNjTT7Gn\nn67eTXT1DZLr7sFiff2V1jEzOvN+tT3cv/h/AAuVqzWuTJe5MlPh8nSZq7MVpkpVpkoVrs760rfr\n+6bL9dcmZirMVqqUKjVmk+brVcrV1QwACcTUyFIhS5WYKjE1ImpEyWsZq5GLIRcFshbImO/LRoGY\nQObavvp6bDWyBOJr274eEfjil7/Cw6P3raLmpTUi2LcDp+ZtjwH7Fh5kZvuB/QDDw8Mr+qCoUKTc\nmYfICFEEUUSIzJdxhJnvt2SbKILIII4h2U8cQRRfW7cohti3LY79tWR/lMsRZXNEuRxxLu8tmyfK\n5cnk8sS5Apl8gUwuTzZXJJMvkk1aLl8km+/w4zK6ASXrWzaO6O/K079w6GWtCqVJmJ2E2QkoX4Xy\nDFSmk+UMlKdvuQzlaWrlGWqVMqE6S61ShmqJUK1AtQTVMlZLWrWMhUqyLBPXyjcveCkBWN6XkBsc\n/XgUuPOD/WbfU274LzSE8CLwIvhwx5V80Bee/w48v5KfFJGmKM/A9KUF7eK89Y89sOfCuzRRD/HS\nJJSnbv8zowxkipAtQKaIZQvEcZ44zkKcg1wO4k5fjzK+jLPeouwi2xlfjzIQxWCRtygGi+ctowXb\ntzo2vn6/GVjM3T03zuTVaI0I9jFgx7ztIeDDBryviLRaCDDzMUyeTdqZ+vLquWT7DEwl4X2rYI6y\nUOyFXBfku711bYH+rvq+a68l+3Jd1wJ70WWswXxLacSf0OvALjMbAcaBXwa+1oD3FZFGK0/D5XG4\nfBIuj81rp5LlOFRnb/y5KAtdm6FrEDZsh60PQ7FvQdt4/Xau069SpeVWHewhhIqZfR14GR/A8Zch\nhMOrrkxEVmbmClw8Chf/Dy4ky4tHvV09t+Bgg+6t0DMEWx+B+5/x7bkQ7xz0ZbFPIb2GNOQ7TQjh\ne8D3GvFeIrJMk+fg7P/C2SNw7gicfddDfGF4d2+DjXfDvU9D307o2ZG0Idiwzfuapa2os0rkTlee\nhtOH4Mw7HuJzbep8/ZhCLwyOwr1fhI33QP89vtw44l0isq4o2EXuJKUpOHMIPnwLPnrLl+fehZCM\nrct1wab74b4vweBuD/PBUe86UVeJJBTsImmp1eD8T+DUj2HsNRh/E869Vw/xjgHY9oiH+LZHYMsn\nvQtFD1eTJSjYRVpldhLG34BTr9XDfOayv1bsg+17/Obl1kc8yDds11W4rIiCXaRZZq7Aif+Aowd8\neeYQhOQBWptGYffPw4593vrvUYhLwyjYRRqlPONX4UcPwLED3rUSqpApwI698PgLHuJDj/oVukiT\nKNhFVqpW9Rucc0F+8lV/honFsP3T8Phvw8iTHuoZPf5YWkfBLrJcIcD59z3Ej/4Ajr9S7yMf3A2P\n/jrc/STsfAwKG1ItVdY3BbvIrVweT4I8uSqf+Mj39wzD6HNw91Mw8oT/dqbIHULBLjLf1EU4/iO/\nIj92AC584Ps7+j3AR570q/K+Ed3slDuWgl3Wt9IUnPyv+lX5R28DAbKdcNdj9e6VwQc0flzWDAW7\nrC/Vso9WmQvysdd8MoYoC0Ofgae+5d0r2x/VM1RkzVKwS3ur1Xz8+LEfejvxnz7ZAwZbHoJ9vwkj\nT8HOz+mZKtI2FOzSXkLwX9M/9kO/Kj/+I58QAqD/E/DQL9ZveHZsTLNSkaZRsMvaFgJcOgbHXvEw\nP/6Kz/ADPnLlvmeSm56P+yNqRdYBBbusLdWKd62cfNVvep58FSZP+2tdW5IQT1rfXamWKpIWBbvc\n2UpXYexgPcjHXvdJkMGvyEeegOF9cNcTMLBLQxBFULDLnaRSgrOHfdTKh28mj7F9N3lwlsHmB+Hh\nr8LwZ721YLZ3kbVIwS7pqJTg/Hs+M9BciJ9+pz6RcnGjDzkc/VkY2gs7PgOFnnRrFlkjFOzSXCH4\nzczTh7xv/Mxhb+ffg1rFj8l2+vPH9+2HbZ/2B2j17lS3isgKKdilMapluHQCLrzvD8q68D6c/8AD\nfOpC/bgNQ7D5AZ+bc/MD3r0ysAuiOL3aRdqMgl2Wb3YSPj45r52Ai0c9yC8dq1+Bg0/rNrDLZwTa\n/KCH+OBujR0XaQEFu7jSlA8bnDjjTzCcPANXxushfukETF+8/mcyRR9SOHi/94UP7IL+XTDwCU0k\nIZKiVQW7mf0S8PvAKLA3hHCwEUVJA9Sq/huXUxf8iYVTFzyYpy54mzwLE6frbfbyje8R56F3GPp2\nwrZP+XrvsPd/9+6EzgH1g4vcgVZ7xX4I+Arw5w2oRcBvNlZLUJ6C8rR3f8xeSdqEz6M5O5G0hfuv\n1EN85jIQbv4ZcR66NkP3Fth0rz+9sHsLdG9N9m/17WKfgltkDVpVsIcQjgDYWvjHH4KPh65VfRmq\nyXrVHxR13fa8ZbWUtPJN1suL7E/Wa8l2ZV5QV2bq6+Upnydzbn3utbkJj5eS7YR8t8/Wk+/21jvs\nQwU7+r0/u6PfA7qjv74v26HAFmlja6uP/cAfwDt/t3ggzw/uhSG93LBspCgLcc4f/5rtgGwxWRZ8\nvbDVJzq+9tr81uGv5bshv+H68M53Q64b4rX11ycirbFkMpjZvwFbbvLSt0MI/7jcDzKz/cB+gOHh\n4WUXeJ2uzT6yIop9wuBry8iXFi14beH23LHRgn2L/OxcKF+3vMl6tMgxuioWkRRYCIv0w97Om5j9\nAHhhuTdP9+zZEw4e1H1WEZHbYWZvhBD2LHWc5voSEWkzqwp2M/sFMxsDPgf8s5m93JiyRERkpVY7\nKuYl4KUG1SIiIg2grhgRkTajYBcRaTMKdhGRNqNgFxFpMwp2EZE205BfULrtDzU7B5xo+Qev3gBw\nPu0iWkzn3P7W2/nC2j3nnSGETUsdlEqwr1VmdnA5v/XVTnTO7W+9nS+0/zmrK0ZEpM0o2EVE2oyC\n/fa8mHYBKdA5t7/1dr7Q5uesPnYRkTajK3YRkTajYF8BM3vBzIKZDaRdS7OZ2R+a2btm9j9m9pKZ\n9aZdU7OY2dNm9p6ZfWBm30q7nmYzsx1m9u9mdsTMDpvZN9KuqVXMLDaz/zazf0q7lmZQsN8mM9sB\n/AxwMu1aWuT7wIMhhE8CPwF+N+V6msLMYuBPgS8Bu4GvmtnudKtqugrwfAhhFPgs8Fvr4JznfAM4\nknYRzaJgv33fAX4HWBc3J0II/xpCqCSbrwJDadbTRHuBD0IIR0MIJeBvgJ9LuaamCiF8FEJ4M1mf\nwINue7pVNZ+ZDQHPAH+Rdi3NomC/DWb2HDAeQng77VpS8hvAv6RdRJNsB07N2x5jHYTcHDO7C/gU\n8ON0K2mJP8YvzlKY4b41NM39AreavBv4PeALra2o+ZYzYbmZfRv/6v7dVtbWQjebeXxdfCszsy7g\n74FvhhCupF1PM5nZs8DZEMIbZvZU2vU0i4J9gRDCT99sv5k9BIwAb5sZeJfEm2a2N4RwuoUlNtxi\n5zzHzH4NeBb4fGjf8bFjwI5520PAhynV0jJmlsVD/bshhH9Iu54WeAx4zsy+DBSADWb21yGEX0m5\nrobSOPYVMrPjwJ4Qwlp8kNCymdnTwB8BT4YQzqVdT7OYWQa/Ofx5YBx4HfhaCOFwqoU1kfkVyl8B\nF0MI30y7nlZLrthfCCE8m3YtjaY+dlnKnwDdwPfN7C0z+7O0C2qG5Abx14GX8ZuIf9vOoZ54DPhV\n4KeSv9u3kitZWeN0xS4i0mZ0xS4i0mYU7CIibUbBLiLSZhTsIiJtRsEuItJmFOwiIm1GwS4i0mYU\n7CIibeb/AS+duLuc5PdCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x110d19da0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = gluon.loss.Logistic()\n",
    "\n",
    "# getting data ready\n",
    "thelabel = nd.ones_like(output)\n",
    "with ag.record():    # start recording\n",
    "    theloss = loss(output, thelabel)\n",
    "theloss.backward()   # and compute the gradient\n",
    "\n",
    "plt.plot(output.asnumpy(), theloss.asnumpy(), label='Logistic loss')\n",
    "plt.plot(output.asnumpy(), output.grad.asnumpy(), label='Gradient of logistic loss')\n",
    "# now compute the loss for y=-1\n",
    "with ag.record():    # start recording\n",
    "    theloss = loss(output, -thelabel)\n",
    "theloss.backward()   # and compute the gradient\n",
    "plt.plot(output.asnumpy(), theloss.asnumpy(), label='Logistic loss for y=-1')\n",
    "plt.plot(output.asnumpy(), output.grad.asnumpy(), label='Gradient of loss for y=-1')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Soft Margin Loss\n",
    "\n",
    "Note that the logistic loss isn't the only loss that one might encounter. For instance, in Support Vector Machines we have a soft-margin loss. It is $0$ whenever data is correctly classified with some confidence, say $y f(x) > 1$. Otherwise we impose a linear penalty. In math this amounts to\n",
    "\n",
    "$$l(y,f) = \\mathrm{max}(0, 1- yf)$$\n",
    "\n",
    "In some cases we want to square this loss function. Quite unsurprisingly, the counterpart to `SoftMargin` is called `SquaredSoftMargin`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlYVeXe//H3zYwTgqA544QjCIgiZjnjcHq0ciwVHEEz\nHzueSq2Tj9qxsiyHMlBRQTNz6FgnczhiOSUOYDjirBkOiKY4AMpw//6w/GWZEwvWZu/v67q6Eln7\nXp+t2w9rr732dyutNUIIIayHndkBhBBCGEuKXQghrIwUuxBCWBkpdiGEsDJS7EIIYWWk2IUQwspI\nsQshhJWRYhdCCCsjxS6EEFbGwYydenp6am9vbzN2LYQQxVZSUtJFrbXXg7Yzpdi9vb1JTEw0Y9dC\nCFFsKaV+epjt5FSMEEJYGSl2IYSwMlLsQghhZUw5xy4sQ05ODqmpqWRnZ5sdRVgwFxcXqlSpgqOj\no9lRxEOSYrdhqamplC5dGm9vb5RSZscRFkhrzaVLl0hNTaVGjRpmxxEPyZBTMUqpskqpFUqpQ0qp\nFKVUiBHrisKVnZ1NuXLlpNTFX1JKUa5cOXlWV8wYdcQ+A1irte6hlHICShi0rihkUuriQeQxUvwU\n+IhdKVUGeBqYB6C1vqW1vlLQde/lx9OXmb3pOPJxfkKI4kZrzZbULUXSX0aciqkJpAMLlFI/KqVi\nlFIl/7iRUipCKZWolEpMT09/rB39e/cZ3l1ziHdWp0i5W4nJkyfTsGFD/Pz88Pf3Z8eOHffdfsuW\nLTRs2BB/f39SUlL4/PPPiyjpX+vSpQtXrhTKsYywElprPkz8kJc2vMS6U+sKfX9GFLsDEAhEaa0D\ngBvA2D9upLWeo7UO0loHeXk98B2x9zSxa0PCQ6ozd8tJxny5l9y8/AIFF+ZKSEhg1apV7N69m717\n9xIfH0/VqlXve5vFixfz6quvkpycTFpaWpEUe25u7n2/v3r1asqWLVvoOUTxlJufy/ht44k7GMcL\n9V4g1Du00PdpRLGnAqla698OtVZwu+gNZ2enmNC1If/brg7LElMZ8flusnPyCmNXogicO3cOT09P\nnJ2dAfD09KRSpUoAbNiwgYCAAHx9fRk0aBA3b94kJiaGZcuWMWnSJPr27cvYsWPZsmUL/v7+TJs2\n7a61N27cSKtWrejVqxc+Pj6MHTuWxYsX06xZM3x9fTl+/DgA33zzDcHBwQQEBNC+fXvS0tIAmDBh\nAhEREYSGhhIWFkZmZia9evXCz8+P3r17ExwcfGcshre3NxcvXuTUqVPUr1+foUOH0rBhQ0JDQ8nK\nyiqqP05hgW7m3eQfG//BV8e+Ynjj4YxrNg47VfhvHyrwi6da6/NKqZ+VUnW11oeBdsDBgke7N6UU\nozv44F7CkYnfHGRQ7C7mhAVRylmu3CyIid8c4ODZq4au2aBSGf7vfxr+5fdDQ0OZNGkSPj4+tG/f\nnt69e9OqVSuys7MZMGAAGzZswMfHh7CwMKKionjllVfYunUrzzzzDD169GDjxo1MnTqVVatW3XP9\nPXv2kJKSgoeHBzVr1mTIkCHs3LmTGTNm8PHHHzN9+nRatmzJ9u3bUUoRExPD+++/z4cffghAUlIS\nW7duxdXVlalTp+Lu7s7evXvZv38//v7+99zn0aNHWbJkCXPnzqVXr158+eWX9OvXr+B/mKLYuZFz\ng1HfjWLH+R2MbTaWvvX7Ftm+jfrRMRJYrJTaC/gD7xi07l8a+GQNPurVmB0nf6Hv3O38cuNWYe9S\nGKxUqVIkJSUxZ84cvLy86N27N7GxsRw+fJgaNWrg4+MDQHh4OJs3b37k9Zs2bUrFihVxdnamVq1a\nhIbefgrs6+vLqVOngNvX8nfs2BFfX18++OADDhw4cOf2Xbt2xdXVFYCtW7fSp08fABo1aoSfn989\n91mjRo07pd+kSZM7+xG25XL2ZYasG0JiWiLvtHynSEsdDLrcUWudDAQZsdajeD6wCmVcHHnp8930\nmp3AosHNqOjmWtQxrML9jqwLk729Pa1bt6Z169b4+voSFxf3l0fDj+q3UzwAdnZ2d762s7O7c958\n5MiRjB49mq5du7Jx40YmTJhw5zYlS/7/awAe9sX63+/T3t5eTsXYoPM3zhO5PpLUa6lMbzOd1lVb\nF3mGYj8rpn2DCiwc1IzzGdn0iErgRPp1syOJh3T48GGOHj165+vk5GSqV69OvXr1OHXqFMeOHQNg\n0aJFtGrV6k+3L126NNeuXStQhoyMDCpXrgxAXFzcX27XsmVLli1bBsDBgwfZt29fgfYrrNOpjFOE\nrQkjLTON6A7RppQ6WEGxAzSvWY4vIpqTnZNHz+gE9p/JMDuSeAjXr18nPDycBg0a4Ofnx8GDB5kw\nYQIuLi4sWLCAnj174uvri52dHcOGDfvT7f38/HBwcKBx48Z/evH0YU2YMIGePXvy1FNP4enp+Zfb\nvfTSS6Snp+Pn58eUKVPw8/PDzc3tsfYprNPBSwcJXxvOzbybzO84n6ZPNDUtizLjevCgoCBdGB+0\ncTz9OmHzdnI1K4eY8CCCa5YzfB/WJCUlhfr165sdo1jIy8sjJycHFxcXjh8/Trt27Thy5AhOTk5m\nRysS8li5v8TziYz8biSlnUozp8McvN28C2U/SqkkrfUDT3tbxRH7b2p5lWL5sBDKl3EmbP5ONqSk\nmR1JWInMzExatmxJ48aNee6554iKirKZUhf3t+nnTQyLH4ZXCS8Wdl5YaKX+KKzuGsFKZV1ZPqwF\nAxbsJGJREh/2bMyzAZXNjiWKudKlS8vHOYo/WXViFf/c+k/qedQjqn0U7i7uZkcCrOyI/TceJZ34\nfGhzmnl78MrSZGJ/OGl2JCGElVmcsphxW8bRpEIT5nWcZzGlDlZa7AClnB1YMLApHRpUYMI3B5ke\nf0TmywghCkxrzafJn/LezvdoW7Utn7b/lJKOfxqPZSqrLXYAF0d7ovoG0qNJFabHH2XiNwfJz5dy\nF0I8nnydz3s73yNqTxTdanXjw9Yf4mzv/OAbFjGrO8f+Rw72drzf3Y+yro7EbD1JRlYO7/fww9He\nqn+mCSEMlpOfw/gfxrPqxCrCGoTxj6B/FMncl8dhmakMZmenePNv9XmtY11W/niGYYuSZHiYBUlL\nS+PFF1+kZs2aNGnShJCQEFauXPnY602YMIGpU6cCMH78eOLj4x9rneTkZFavXv3YOYT1yM7N5u/f\n/51VJ1bxvwH/y6tBr1psqYONFDvcHh42ok1t3n62Ed8dvkDY/J1czc4xO5bN01rz7LPP8vTTT3Pi\nxAmSkpL44osvSE1NvWu7B43O/SuTJk2iffv2j3VbKXYBcO3WNSLXR7I5dTP/DP4nQ/2GWvynStlM\nsf+mf/PqzOgTwO6fLvPCnO1cvH7T7Eg27bvvvsPJyemud5ZWr16dkSNHEhsbS8+ePfmf//kfQkND\nuX79Ou3atSMwMBBfX1++/vrrO7eZPHkydevWpX379hw+fPjO7w8YMIAVK1YAt6c1tmrViiZNmtCx\nY0fOnTsHQOvWrRkzZgzNmjXDx8eHLVu2cOvWLcaPH8/SpUvx9/dn6dKlRfQnIizJxayLDFo3iL3p\ne5ny9BR61+ttdqSHYvXn2O+la+NKlHFxYNhnSfSMvj08rIq7jX9M65qxcN7g+SdP+ELn9+67yYED\nBwgM/Ovx/QkJCezduxcPDw9yc3NZuXIlZcqU4eLFizRv3pyuXbuye/duvvjiC3788Udyc3MJDAyk\nSZMmd62Tk5PDyJEj+frrr/Hy8mLp0qW8+eabzJ8/H7j9jGDnzp2sXr2aiRMnEh8fz6RJk0hMTOST\nTz4p+J+FKHbOXj9LxPoI0m6k8XG7j2lZuaXZkR6aTRY7QOu65flscDCDYnfRI+p2udepUNrsWDZv\nxIgRbN26FScnJ0aMGEGHDh3w8PAAbp+2eeONN9i8eTN2dnacOXOGtLQ0tmzZwnPPPUeJErd/OHft\n2vVP6x4+fJj9+/fToUMH4PaIgIoVK975/vPPPw/IqF1x2/Erx4lYH0FWbhZzQucQUD7A7EiPxGaL\nHSDI24OlkSH0n7eTXrMTiB3YjMZVbfQjzh5wZF1YGjZsyJdffnnn61mzZnHx4kWCgm6Pw/j96NzF\nixeTnp5OUlISjo6OeHt7k52dDfDAc55aaxo2bEhCQsI9v//buF17e/vHPp8vrMP+i/sZHj8ce2XP\ngo4LqOtR1+xIj8zmzrH/Uf2KZfhyeAilXBx4ce52th27aHYkm9K2bVuys7OJioq683uZmZn33DYj\nI4Py5cvj6OjI999/z08//QTA008/zcqVK8nKyuLatWt88803f7pt3bp1SU9Pv1PsOTk5d32oxr0Y\nMRZYFC/bz21n8LrBlHQsyaLOi4plqYMUOwDVy5VkxbAWVHZ3ZcCCXaw7cN7sSDZDKcVXX33Fpk2b\nqFGjBs2aNSM8PJwpU6b8adu+ffuSmJhIUFAQixcvpl69egAEBgbSu3dv/P396d69O0899dSfbuvk\n5MSKFSsYM2YMjRs3xt/fn23btt03W5s2bTh48KC8eGojNvy0gZfiX6JSqUos7LyQqmXu/8Hqlsyq\nxvYW1JXMWwyM3cWen6/wXnc/egUV37/YhyGjWMXDsvbHysqjK5mQMIFGno34tN2nuDlb5qx9mxzb\nW1BlSzixeEgwT9b25PUVe5m7+YTZkYQQhSzuQBzjt42necXmzO0w12JL/VEYUuxKqVNKqX1KqWSl\nlOUdij+CEk4OxIQH8TffikxencIH6w7J8DAhrJDWmpm7ZzI1cSqh1UP5uO3HlHC0jsuejbwqpo3W\n2ipeeXR2sGfmCwGUcXVg1vfHuZyZw9vdGmFvZ9nvNhNCPJy8/Dwm75jM8iPL6V6nO281fwt7O3uz\nYxnGpi93vB97O8U7z/lStoQTURuPczUrh496+ePkIGevhCjOcvJyGLd1HOtOrWNwo8GMChxl8SMC\nHpVRxa6B/yqlNDBbaz3njxsopSKACIBq1aoZtNvCpZRiTKd6lHV15N01h7ianUt0v0BKOMnPQyGK\no8ycTEZvHM0PZ39gdJPRDGw00OxIhcKow88ntdaBQGdghFLq6T9uoLWeo7UO0loHeXl5GbTbohHZ\nqhZTuvuy9Wg6/WJ2kJEpw8OEKG4ybmYQsT6ChHMJTGwx0WpLHQwqdq312V//fwFYCTQzYl1L0rtp\nNT7tG8j+M1fpPSeBC1ezzY5kFSZPnkzDhg3x8/PD39+fHTt23Hf7LVu20LBhQ/z9/UlJSeHzzz8v\noqR/rUuXLly5cuWht09PTyc4OJiAgAC2bNlSiMke7JNPPqF27doopbh40SpeIrun9Mx0Bq4byMFL\nB/mw1Yc8X+d5syMVqgIXu1KqpFKq9G+/BkKB/QVd1xJ1alSR+QOacvqXTHpEJ3D60r3fISkeTkJC\nAqtWrWL37t3s3buX+Ph4qla9/3sHFi9ezKuvvkpycjJpaWlFUuwPGjGwevVqypZ9+FEUGzZsoF69\nevz444/3fDPVveTlFc7nBzz55JPEx8dTvXr1QlnfEvx87WfC1oSRei2VWe1m0b76441xLk6MOGKv\nAGxVSu0BdgLfaq3XGrCuRWpZx5PFQ4K5mp1Dj+htHDp/1exIxda5c+fw9PS8M6fF09OTSpUqAbfL\nLyAgAF9fXwYNGsTNmzeJiYlh2bJlTJo0ib59+zJ27Fi2bNmCv78/06ZNu2vtjRs30qpVK3r16oWP\njw9jx45l8eLFNGvWDF9fX44fPw7AN998c+fouX379qSlpQG3P6wjIiKC0NBQwsLCyMzMpFevXvj5\n+dG7d2+Cg4P57U123t7eXLx4kVOnTlG/fn2GDh1Kw4YNCQ0NJSsr665cycnJvP7666xevRp/f3+y\nsrJYsmQJvr6+NGrUiDFjxtzZtlSpUowfP57g4OC7ZtwcP378romYR48e/dM0y4cVEBCAt7f3Y922\nODhy+Qhha8K4lnONmNAYQiqFmB2pSBT4VUCt9QmgsQFZio2Aau4siwyh/7wd9IpOYMHAZjSpbjmf\nUP44puycwqFfDhm6Zj2PeoxpNuYvvx8aGsqkSZPw8fGhffv29O7dm1atWpGdnc2AAQPYsGEDPj4+\nhIWFERUVxSuvvMLWrVt55pln6NGjBxs3bmTq1KmsWrXqnuvv2bOHlJQUPDw8qFmzJkOGDGHnzp3M\nmDGDjz/+mOnTp9OyZUu2b9+OUoqYmBjef/99PvzwQ+D2/PatW7fi6urK1KlTcXd3Z+/evezfvx9/\nf/977vPo0aMsWbKEuXPn0qtXL7788kv69et35/v+/v53jQM+e/YsY8aMISkpCXd3d0JDQ/nqq694\n9tlnuXHjBo0aNWLSpEl37aNWrVq4ubmRnJyMv78/CxYsYMCAAQD8/e9/5/vvv/9Trj59+jB27Nj7\n/n1Zm+QLyby04SVc7V2J7RhLbffaZkcqMnLt3mPyqVCaFcNa4FHSiX4xO9h0JN3sSMVOqVKlSEpK\nYs6cOXh5edG7d29iY2M5fPgwNWrUwMfHB4Dw8HA2b978yOs3bdqUihUr4uzsTK1atQgNDQXA19f3\nzmje1NRUOnbsiK+vLx988MFdg8G6du2Kq6srAFu3bqVPnz4ANGrUCD8/v3vus0aNGndK/2FGAO/a\ntYvWrVvj5eWFg4MDffv2vXNf7e3t6d69+z1vN2TIEBYsWEBeXh5Lly7lxRdfBGDatGkkJyf/6T9b\nK/UfzvxAxPoI3J3dWdhloU2VOsh17AVS1aMEy4e1IHz+TobE7WJab3+e8atkdqzHcr8j68Jkb29P\n69atad26Nb6+vsTFxf3l0fCj+u0UD4Cdnd2dr+3s7O6cNx85ciSjR4+ma9eubNy4kQkTJty5ze9H\nBj/su49/v097e/s/nYr5o/ut6+Ligr39vd800717dyZOnEjbtm1p0qQJ5cqVAx58xN6xY0fS0tII\nCgoiJibmYe5SsbP21FrGbRlH7bK1iWofhaerp9mRipwUewF5lXZmSURzhsTtYuSSH7malcuLwcXj\nOn2zHT58GDs7O+rUqQPcPv9cvXp16tWrx6lTpzh27Bi1a9dm0aJFtGrV6k+3N2KsbkZGBpUrVwYg\nLi7uL7dr2bIly5YtuzPxcd8+Yz5tKjg4mFGjRnHx4kXc3d1ZsmQJI0eOfODtXFxc6NixI8OHD2fe\nvHl3fv+PrzX80bp16wqc2ZItP7KctxPeJqB8AB+3+5gyTmXMjmQKORVjADdXRxYOCqa1jxdvrNzH\npxuPyXyZh3D9+nXCw8Np0KABfn5+HDx4kAkTJuDi4sKCBQvo2bMnvr6+2NnZ3fWZqL/x8/PDwcGB\nxo0bP7DQ/sqECRPo2bMnTz31FJ6ef31k99JLL5Geno6fnx9TpkzBz88PN7eCD4uqWLEi7777Lm3a\ntKFx48YEBgbSrVu3h7pt3759UUrdOcX0OGbOnEmVKlVITU3Fz8+PIUOGPPZaZtJaE7MvhkkJk2hZ\nuSXRHaJtttRBxvYaKicvn1eX7+Hr5LNEPF2TcZ3rWfRbla19FKuR8vLyyMnJwcXFhePHj9OuXTuO\nHDmCk5OTaZmmTp1KRkYGb7/9dqHvy5IfK1prPkz8kLiDcXSp0YV/tfwXjnaOZscqFA87tldOxRjI\n0d6Oab38KevqyJzNJ7iSeYt3nvPFwV6eGBV3mZmZtGnThpycHLTWREVFmVrqzz33HMePH+e7774z\nLYMlyM3PZVLCJFYeW8kL9V5gbLOx2Cn59ybFbjA7O8WErg1xK+HEzA1HycjKYUafAFwcrWdynC0q\nXbo0lvQsc+XKlWZHMN3NvJuM2TyGDac3MKzxMF5q/JJFP0MuSvKjrRAopRjdwYfxzzRg3YE0BsXu\n4vpNy/yAZHktQDyIJT5GbuTcYET8CDac3sCYpmMY4T9CSv13pNgL0aCWNfiwZ2N2nPyFvnO3c/nG\nLbMj3cXFxYVLly5Z5D9cYRm01ly6dAkXFxezo9xxOfsyQ9YNITEtkcktJ9OvQb8H38jGyKmYQta9\nSRXKuDoy4vPd9JydwKLBzajo5mp2LIA7V0Okp8ubq8Rfc3FxoUqVKmbHAOD8jfNEro8k9Voq01pP\no021NmZHskhyVUwR2X7iEkPiEnFzdWTR4GbU9CpldiQhipVTGaeIWB/B1VtX+bjtxzR9oqnZkYqc\nfJi1hWlesxxfRDQnOyePntEJ7D+TYXYkIYqNlEsphK8N52beTeZ3nG+Tpf4opNiLUKPKbiwbFoKz\ngx0vzNnOzpO/mB1JCIuXlJbEoHWDcLJ3IrZTLA3KNTA7ksWTYi9itbxKsWJ4C8qXcab/vB18dyjN\n7EhCWKxNP28icn0kXiW8WNR5ETXcapgdqViQYjdBpbKuLIsMoe4TpRm6MImvfjxjdiQhLM6qE6sY\n9f0oapetTWynWJ4o+YTZkYoNKXaTlCvlzOdDm9PM24NXliYT+8NJsyMJYTEWpyxm3JZxNKnQhHkd\n5+Hh4mF2pGJFit1EpZwdWDCwKR0aVGDCNweZHn9ErikXNk1rTVRyFO/tfI82VdvwaftPKelY8sE3\nFHeRYjeZi6M9UX0D6dGkCtPjjzLxm4Pk50u5C9uTr/OZsmsKn+75lG61uvFR649wtnd+8A3Fn8gb\nlCyAg70d73f3w83VkXlbT5KRlcP7PfxwlOFhwkbk5Ocw/ofxrDqxiv4N+vNq0KsyzKsADCt2pZQ9\nkAic0Vo/Y9S6tsLOTvHPv9XHvYQjU/97hGvZOXzyYqAMDxNWLzs3m1c3vcqm1E2MDBjJUN+hMvel\ngIz8kTgKSDFwPZujlOLltnV4+9lGbDh0gbD5O7manWN2LCEKzbVb1xgWP4zNqZv5Z/A/ifCLkFI3\ngCHFrpSqAvwNsM4PUSxi/ZtXZ0afAHb/dJkX5mzn4vWbZkcSwnCXsi4xeN1g9lzYw5Snp9C7Xm+z\nI1kNo47YpwOvA/kGrWfzujauxNzwII6nX6dXdAKplzPNjiSEYc5eP8uAtQM4mXGSmW1n0rlGZ7Mj\nWZUCF7tS6hnggtY66QHbRSilEpVSiTJN8OG0qVuezwYHk379Jj2jEzh2oWAf3CyEJThx5QRha8K4\nlHWJOaFzeKrKU2ZHsjpGHLE/CXRVSp0CvgDaKqU+++NGWus5WusgrXWQl5eXAbu1DUHeHiyLDCEn\nT9MzOoE9P18xO5IQj23/xf2Erw0nNz+XBZ0WEFA+wOxIVqnAxa61Hqe1rqK19gb6AN9prWXyvYHq\nVyzDl8NDKOXiwItzt7Pt2EWzIwnxyHac28HgdYMp6ViShZ0XUtejrtmRrJZcKFpMVC9XkhXDWlDZ\n3ZUBC3ax7sB5syMJ8dA2nN7A8PjhVCpViYWdF1KtTDWzI1k1Q4tda71RrmEvPBXKuLAsMoSGlcsw\n/LMkliX+bHYkIR5o5dGVjN44mvrl6hPbKZbyJcqbHcnqyRF7MVO2hBOfDQ7mydqevL5iLzFbTpgd\nSYi/FHcgjvHbxhP8RDBzO8zFzdnN7Eg2QYq9GCrp7EBMeBB/863Iv75NYeq6wzI8TFgUrTUzd89k\nauJUOlTvwCftPqGEYwmzY9kMmRVTTDk72DPzhQDKuDrwyffHuJx5i0ndGmFvJ+/aE+bKy89j8o7J\nLD+ynO51uvNW87ewt5PRGEVJir0Ys7dTvPOcL2VLOBG18TgZWTl81MsfJwd5IibMkZOXwxtb32Dt\nqbUMbjSYUYGjZESACaTYizmlFGM61aOsqyPvrjnEtexcovoFUsJJ/mpF0crMyWT0ptH8cOYHRjcZ\nzcBGA82OZLPk0M5KRLaqxXvP+7LlaDr95+0kI1OGh4mik3Ezg8j1kSScTWBCyAQpdZNJsVuRPs2q\nMevFQPalZtB7TgIXrmabHUnYgPTMdAauG8iBSweY2moq3X26mx3J5kmxW5nOvhWZP6App3/JpEd0\nAqcvyfAwUXh+vvYzYWvCSL2Wyqx2s+hQvYPZkQRS7FapZR1PFg8J5mp2Dj2it3Ho/FWzIwkrdOTy\nEcLXhHMt5xoxoTGEVAoxO5L4lRS7lQqo5s6yyBCUgl7RCST9dNnsSMKKJF9IZsDaASgUsR1j8fPy\nMzuS+B0pdivmU6E0K4a1wKOkE/1idrD5iIxLFgW37cw2ItZH4O7szsIuC6ntXtvsSOIPpNitXFWP\nEiwf1gJvz5IMjtvFt3vPmR1JFGNrT61lxHcjqFa6GnGd46hcqrLZkcQ9SLHbAK/SznwR0Rz/qmV5\necluPt9x2uxIohhafmQ5r296HT9PP+Z3mo+nq6fZkcRfkGK3EW6ujiwcFExrHy/eWLmPTzcek/ky\n4qForYnZF8OkhEm0rNyS6A7RlHEqY3YscR9S7DbE1cmeOWFBdPOvxPtrD/PumkNS7uK+tNZ8lPQR\nM3bPoEuNLsxoOwNXB1ezY4kHkPed2xhHezum9fLHzdWROZtPcCXzFu8854uDvfyMF3fLzc9lUsIk\nVh5bSZ+6fRgXPA47JY+T4kCK3QbZ2Skmdm1I2RJOzNxwlKtZucx4wR9nB5nAJ267mXeTsZvHEn86\nnki/SEb4j5BhXsWI/Pi1UUopRnfwYfwzDVh74DyDYndx/Wau2bGEBbiRc4MR8SOIPx3PmKZjeDng\nZSn1YkaK3cYNalmDD3s2ZvuJX+g7dzuXb9wyO5Iw0ZXsKwxZN4TEtEQmt5xMvwbyufTFUYGLXSnl\nopTaqZTao5Q6oJSaaEQwUXS6N6lCdL8mpJy/Rs/ZCZzLyDI7kjDB+RvnCV8bzpHLR5jWehpda3U1\nO5J4TEYcsd8E2mqtGwP+QCelVHMD1hVFqEODCsQNbMb5jGx6RCVw8uINsyOJIvTT1Z8IXxNOWmYa\n0R2iaVOtjdmRRAEUuNj1bdd//dLx1//kGrpiKKRWOZYMbU5WTh49o7dx4GyG2ZFEEUi5lELYmjCy\ncrOY13EeTZ9oanYkUUCGnGNXStkrpZKBC8B6rfUOI9YVRc+3ihvLh4XgZG9Hn9nb2XnyF7MjiUKU\nlJbEoHWDcLJ3IrZzLA3LNTQ7kjCAIcWutc7TWvsDVYBmSqlGf9xGKRWhlEpUSiWmp8swKktWy6sU\nK4a3oHw5u3w6AAAW/UlEQVQZZ/rP28F3h9LMjiQKwebUzUSuj8TT1ZOFnRZS062m2ZGEQQy9KkZr\nfQXYCHS6x/fmaK2DtNZBXl5eRu5WFIJKZV1ZFhmCT4XSRCxM4qsfz5gdSRho1YlVjPpuFLXK1iKu\ncxwVS1U0O5IwkBFXxXgppcr++mtXoD1wqKDrCvOVK+XM50ODCfJ255WlycRtO2V2JGGAz1M+Z9yW\ncQRUCGBe6Dw8XDzMjiQMZsQRe0Xge6XUXmAXt8+xrzJgXWEBSrs4EjuwGR0aVOD//nOAGfFHZb5M\nMaW1JmpPFO/ufJfWVVsT1T6KUk6lzI4lCkGBRwporfcCAQZkERbKxdGeqL6BjP33PqbFH+Fy5i3G\nP9MAOzt5N2Jxka/zeX/X+yxOWUzXWl2Z2GIiDnYyUcRayd+seCgO9na8390PN1dH5m09SUZWDu/3\n8MNRhodZvJz8HMb/MJ5VJ1bRr34/Xmv6mgzzsnJS7OKh2dkp/vm3+riXcGTqf49wLTuHT14MxMVR\nhodZquzcbF7b9BobUzcyMmAkQ32HytwXGyA/tsUjUUrxcts6vP1sIzYcukDY/J1czc4xO5a4h2u3\nrjEsfhibUjfxz+B/EuEXIaVuI6TYxWPp37w603v7s/uny7wwZzsXr980O5L4nUtZlxi8bjB7Luzh\nvafeo3e93mZHEkVIil08tm7+lZkbHsTx9Ov0ik4g9XKm2ZEEcPb6WQasHcDJjJPMaDuDLjW7mB1J\nFDEpdlEgbeqW57PBwaRfv0nP6ASOXbhmdiSbduLKCcLWhHEp6xKzO8zm6SpPmx1JmECKXRRYkLcH\nSyNCyMnT9IxOYG/qFbMj2aQDFw8Qvjac3PxcFnRaQGCFQLMjCZNIsQtDNKhUhhXDQijp7MALc7az\n7fhFsyPZlJ3ndjJo3SBKOpZkYeeF1PWoa3YkYSIpdmEYb8+SfDm8BZXdXRmwYBfrDpw3O5JN2HB6\nA8Pjh1OpVCXiOsVRrUw1syMJk0mxC0NVKOPCssgQGlYqw/DPklie+LPZkazaV8e+YvTG0dTzqEds\np1gqlKxgdiRhAaTYheHKlnDis8HBPFnbk9dW7CVmywmzI1mluANxvPXDWwQ/Eczc0Lm4ObuZHUlY\nCCl2UShKOjsQEx5EF98n+Ne3KUxdd1iGhxlEa83M3TOZmjiVDtU78Em7TyjhWMLsWMKCyEgBUWic\nHez5+IVAyrjs45Pvj3E58xaTujXCXoaHPba8/Dze2fEOy44so3ud7rzV/C3s7WSkg7ibFLsoVPZ2\ninef96VsCSeiNx0nIyuHj3r54+QgTxYfVU5eDm9sfYO1p9YyqNEgXgl8RUYEiHuSYheFTinF2M71\ncC/hyLtrDnEtO5fofk1wdZIjzYeVmZPJ6E2j+eHMD/y9yd8Z1GiQ2ZGEBZPDJlFkIlvV4r3nfdly\nNJ1+83aQkSnDwx5Gxs0MItdHknA2gQkhE6TUxQNJsYsi1adZNWa9GMi+1Ax6z0ngwtVssyNZtPTM\ndAauG8iBSweY2moq3X26mx1JFANS7KLIdfatyPwBTTn9SyY9ohM4fUmGh93Lz9d+JmxNGKnXUpnV\nbhYdqncwO5IoJqTYhSla1vFk8ZBgrmbn0CN6G4fPy/Cw3zt6+Sjha8K5eusqMaExhFQKMTuSKEak\n2IVpAqq5sywyBKWg1+wEdp++bHYki5B8IZkBawegUMR1isPPy8/sSKKYKXCxK6WqKqW+V0qlKKUO\nKKVGGRFM2AafCqVZMawF7iUc6Tt3B5uPpJsdyVTbzmwjYn0EZZ3LEtc5jtrutc2OJIohI47Yc4F/\naK3rA82BEUqpBgasK2xEVY8SLB/WAm/PkgyO28W3e8+ZHckU606tY8R3I6hWuhpxneOoUrqK2ZFE\nMVXgYtdan9Na7/7119eAFKByQdcVtsWrtDNfRDSncZWyvLxkN0t2njY7UpFacWQFr216DV9PX+Z3\nmo+nq6fZkUQxZug5dqWUNxAA7LjH9yKUUolKqcT0dNt+ui3uzc3VkUWDg2nt48W4f+8jauNxsyMV\niXn75jExYSJPVn6S2R1mU8apjNmRRDFnWLErpUoBXwKvaK2v/vH7Wus5WusgrXWQl5eXUbsVVsbV\nyZ45YUF086/ElLWHeHd1itUOD9Na81HiR0zfPZ3ONTozs81MXB1czY4lrIAhIwWUUo7cLvXFWut/\nG7GmsF2O9nZM6+WPm6sjszef4EpmDpOfa4SDvfVcxJWbn8ukhEmsPLaS3nV780bwG9gp67l/wlwF\nLnZ1ewrRPCBFa/1RwSMJAXZ2ioldG1LW1ZGZ3x0jIyuHGS/44+xQ/OfL3Mq7xZjNY4g/HU+kXyQj\n/EfIMC9hKCMOEZ4E+gNtlVLJv/7XxYB1hY1TSjE6tC5vPdOAtQfOMyh2F9dv5podq0Bu5NzgpQ0v\nEX86ntebvs7LAS9LqQvDFfiIXWu9FZBHpig0g1vWoKyrI69/uZe+MTuIHdAU95JOZsd6ZFeyr/DS\nhpc4eOkgk1tOpmutrmZHElZKTuqJYqF7kypE92tCyrmr9JqdwPmM4jU8LO1GGgPWDuDwL4eZ1nqa\nlLooVFLsotjo0KACcQObcS4jm+5R2zh58YbZkR7KT1d/ImxNGOczzxPdIZo21dqYHUlYOSl2UayE\n1CrHkqHNycrJo2f0Ng6czTA70n0d+uUQYWvCyMrNYl7HeTR9oqnZkYQNkGIXxY5vFTeWRYbgZG9H\nn9nb2XnyF7Mj3VNSWhID1w7Eyd6J2M6xNCzX0OxIwkZIsYtiqXb5Uiwf3gKvMs70n7eD7w6lmR3p\nLptTNxO5PhJPV08WdlpITbeaZkcSNkSKXRRblcu6sjwyBJ8KpYlYmMTXyWfMjgTAtye+ZdR3o6jp\nVpO4znFULFXR7EjCxkixi2KtXClnPh8aTJC3O68sTWZhwilT8yw5tIRxW8bhX96f+R3n4+HiYWoe\nYZuk2EWxV9rFkdiBzWhXrwLjvz7AjPijRT5fRmtN1J4o3tnxDq2qtiK6QzSlnEoVaQYhfiPFLqyC\ni6M90f0C6R5YhWnxR5j4zUHy84um3PN1PlN2TeHT5E/pWqsr01pPw9neuUj2LcS9GDIETAhL4GBv\nxwc9/HBzdWT+Dye5mpXDlB5+OBbi8LCc/Bz+74f/45sT39Cvfj9ea/qaDPMSppNiF1bFzk7x1jP1\n8SjpyNT/HuFqdg6fvBiIi6Pxw8Oyc7N5bdNrbEzdyMv+LxPhFyFzX4RFkEMLYXWUUrzctg5vP9uI\nDYcuED5/J1ezcwzdx7Vb1xgeP5xNqZt4M/hNIhtHSqkLiyHFLqxW/+bVmd7bn6SfLvPCnO1cvH7T\nkHUvZV1i8LrBJF9I5r2n3qNPvT6GrCuEUaTYhVXr5l+ZuWFBHE+/Tq/oBM5cySrQeueun2PA2gGc\nzDjJjLYz6FJTJlQLyyPFLqxem3rlWTQ4mPTrN+kRtY1jF64/1jonMk7Qf01/LmVdYnaH2Txd5WmD\nkwphDCl2YROaenuwNCKEnDxNr9kJ7E298ki3P3DxAAPWDCA3P5cFnRYQWCGwkJIKUXBS7MJmNKhU\nhhXDQijhZM8Lc7az7fjFh7rdznM7GbRuECUcS7Cw80LqetQt5KRCFIwUu7Ap3p4lWTGsBZXdXRmw\nYBfrDpy/7/YbTm9gePxwKpWqRFynOKqVqVZESYV4fFLswuY84ebCssgQGlQsw/DPklie+PM9t/vq\n2FeM3jiaeh71iO0US4WSFYo4qRCPx5BiV0rNV0pdUErtN2I9IQpb2RJOLB4SzJO1PXltxV5itpy4\n6/sLDyzkrR/eotkTzZgbOhc3ZzeTkgrx6Iw6Yo8FOhm0lhBFoqSzAzHhQXTxfYJ/fZvC1HWHyc/P\nZ+bumXyQ+AEdqndgVrtZlHAsYXZUIR6JISMFtNablVLeRqwlRFFydrDn4xcCKeOyj0++P8KmX6I5\ndSue5+s8z/jm47G3M34UgRCFrchmxSilIoAIgGrV5AUoYTns7RSTutVjf84sTt3cRlW7LrzRVEpd\nFF9F9uKp1nqO1jpIax3k5eVVVLsV4oGycrP4343/y083txHiHsbBA08TsSiJrFt5ZkcT4rHIVTHC\npmXczCDivxEknE1gQsgE5nR9jfee92XL0XT6zdtBRqaxw8OEKApS7MJmXcy6yKB1g9h/aT8fPP0B\n3X26A9CnWTVmvRjIvtQMes9J4MK1bJOTCvFojLrccQmQANRVSqUqpQYbsa4QhSX1Wipha8L4+drP\nzGo3i1Dv0Lu+39m3IvMHNOX0L5n0jE7g518yTUoqxKMzpNi11i9orStqrR211lW01vOMWFeIwnD0\n8lHC1oSRcTODuaFzaVGpxT23a1nHk8VDgrmSmUP3qG0cPn+tiJMK8XjkVIywKXvS9zBg7QAAYjvF\n0tir8X23D6jmzvJhISgFvWYnsPv05SJIKUTBSLELm7Ht7DaG/ncobs5uLOy8kDrudR7qdj4VSrNi\nWAvcSzjSd+4OthxNL+SkQhSMFLuwCf899V9GbBhB1dJVWdh5IVVKV3mk21f1KMHyYS3w9izJoNhd\nrN53rpCSClFwUuzC6q04soJXN72Kr6cvCzotwNPV87HW8SrtzBcRzWlcpSwvf76bJTtPG5xUCGNI\nsQurNm/fPCYmTOTJyk8yu8NsyjiVKdB6bq6OLBoczNM+Xoz79z6iNh43KKkQxpFiF1ZJa81HiR8x\nffd0OtfozMw2M3F1cDVkbVcne+aGBdHNvxJT1h7i3dUpaK0NWVsIIxTZrBghikpefh6Ttk/i30f/\nTe+6vXkj+A3slLHHMI72dkzr5Y+bqyOzN5/gSmYO7zzvi72dMnQ/QjwOKXZhVW7l3WLslrGs/2k9\nEX4RvOz/MkoVTtna2Skmdm1IWVdHZn53jKvZOUzv44+zgwwPE+aSYhdWIzMnk1Hfj2L7ue283vR1\n+jfoX+j7VEoxOrQubiWceHvVQa7FJjK7fxNKOss/LWEeOccurMKV7CsM+e8Qdp3fxb+e/FeRlPrv\nDW5Zgw97NibhxCVejNnB5Ru3inT/QvyeFLso9tJupDFg7QAO/3KYj1p/RLfa3UzJ0b1JFaL7NSHl\n3FV6zU7gfIYMDxPmkGIXxdpPV38ibE0Y5zPPE90hmrbV2pqap0ODCsQNbMa5jGy6R23j5MUbpuYR\ntkmKXRRbh345RNiaMLJys5jXcR5Nn2hqdiQAQmqVY8nQ5mTl5NEzehsHzmaYHUnYGCl2USztTtvN\noLWDcLRzJLZzLA3LNTQ70l18q7ixLDIEJ3s7+szZzq5Tv5gdSdgQKXZR7GxO3Uzk+kjKuZZjUedF\n1HSraXake6pdvhTLh7fAq7Qz/eft4PtDF8yOJGyEFLsoVr498S2jvhtFDbcaxHWOo2KpimZHuq/K\nZV1ZHhlCnfKlGbowka+Tz5gdSdgAKXZRbCw5tIRxW8bhX96f+R3n4+HiYXakh1KulDOfDw0myNud\nV5YmszDhlNmRhJWTYhcWT2tN9J5o3tnxDq2qtiKqfRSlnEqZHeuRlHZxJHZgM9rVq8D4rw8wc8NR\nmS8jCo0Uu7Bo+Tqf93e9z6zkWXSt1ZVprafh4uBidqzH4uJoT3S/QLoHVuGj9UeYtOog+flS7sJ4\nhrzvWSnVCZgB2AMxWuv3jFhX2Lac/BwmbJvAf47/h371+/Fa09cMH+ZV1Bzs7fighx9uro7M/+Ek\nGVk5TOnuh6N98b5fwrIUuNiVUvbALKADkArsUkr9R2t9sKBrC9uVnZvNa5teY2PqRkb4jyDSL7LQ\nhnkVNTs7xVvP1Me9hCMfrj/C1awcPnkxEBdHGR4mjGHEEXsz4JjW+gSAUuoLoBtgeLFn5mRyK09m\ncFi7m3k3GbtlLElpSbwZ/CZ96vUxO5LhlFKMbFeHsiWdGP/1fsLn72TGszVwkW63eiVKl8XJuXBP\nJxpR7JWBn3/3dSoQbMC6f/JR0kcsPby0MJYWFsZBOfDeU+/RpWYXs6MUqv7Nq1PGxYEtKz7miago\ns+OIIrC31Tz82vQo1H0YUez3en78p1eElFIRQARAtWrVHmtHHb07UsOtxmPdVhQvfp5++Hr5mh2j\nSHTzr0yz066wG3bUfQ0t1zRYteo1C/9xbUSxpwJVf/d1FeDsHzfSWs8B5gAEBQU91qUATZ9oajHz\nQIQwUsXSTgAE93kTrOS1BGEeIw4NdgF1lFI1lFJOQB/gPwasK4Tt0HmAklIXhijwEbvWOlcp9TKw\njtuXO87XWh8ocDIhbInOBzt55VQYw5Dr2LXWq4HVRqwlhE3Kz4Nifo2+sBzySBLCEug8UHLELowh\nxS6EJciXUzHCOFLsQlgCOWIXBpJiF8IS6Hy5IkYYRopdCEuQnyenYoRhpNiFsARyKkYYSIpdCEsg\nR+zCQFLsQlgCreU6dmEYeSQJYQnkVIwwkBS7EJYgPw/s5J+jMIY8koSwBHLELgwkxS6EJZAhYMJA\nUuxCWAIZAiYMJI8kISyBzpdTMcIwUuxCWAJ58VQYSB5JQlgCOWIXBpJiF8ISaDnHLowjjyQhLIGM\nFBAGkmIXwhLIdezCQAUqdqVUT6XUAaVUvlIqyKhQQtgc+QQlYaCCHrHvB54HNhuQRQjbpfPlHLsw\njENBbqy1TgFQ8skvQhSMzgPlaHYKYSUKVOxFbtMHsH+F2SmEMN7lU1C9hdkphJV4YLErpeKBJ+7x\nrTe11l8/7I6UUhFABEC1atUeOuBdSpUHr7qPd1shLJlXXfDtaXYKYSWU1rrgiyi1EXhVa534MNsH\nBQXpxMSH2lQIIcSvlFJJWusHXqgir9YIIYSVKejljs8ppVKBEOBbpdQ6Y2IJIYR4XAW9KmYlsNKg\nLEIIIQwgp2KEEMLKSLELIYSVkWIXQggrI8UuhBBWRopdCCGsjCFvUHrknSqVDvxU5DsuOE/gotkh\nipjcZ+tna/cXiu99rq619nrQRqYUe3GllEp8mHd9WRO5z9bP1u4vWP99llMxQghhZaTYhRDCykix\nP5o5Zgcwgdxn62dr9xes/D7LOXYhhLAycsQuhBBWRor9MSilXlVKaaWUp9lZCptS6gOl1CGl1F6l\n1EqlVFmzMxUWpVQnpdRhpdQxpdRYs/MUNqVUVaXU90qplF8/lH6U2ZmKilLKXin1o1JqldlZCoMU\n+yNSSlUFOgCnzc5SRNYDjbTWfsARYJzJeQqFUsoemAV0BhoALyilGpibqtDlAv/QWtcHmgMjbOA+\n/2YUkGJ2iMIixf7opgGvAzbx4oTW+r9a69xfv9wOVDEzTyFqBhzTWp/QWt8CvgC6mZypUGmtz2mt\nd//662vcLrrK5qYqfEqpKsDfgBizsxQWKfZHoJTqCpzRWu8xO4tJBgFrzA5RSCoDP//u61RsoOR+\no5TyBgKAHeYmKRLTuX1wlm92kMJSoA/asEb3+/Bu4A0gtGgTFb6H+cBypdSb3H7qvrgosxUhdY/f\ns4lnZUqpUsCXwCta66tm5ylMSqlngAta6ySlVGuz8xQWKfY/0Fq3v9fvK6V8gRrAHqUU3D4lsVsp\n1Uxrfb4IIxrur+7zb5RS4cAzQDttvdfHpgJVf/d1FeCsSVmKjFLKkdulvlhr/W+z8xSBJ4GuSqku\ngAtQRin1mda6n8m5DCXXsT8mpdQpIEhrXRwHCT00pVQn4COgldY63ew8hUUp5cDtF4fbAWeAXcCL\nWusDpgYrROr2EUoc8IvW+hWz8xS1X4/YX9VaP2N2FqPJOXbxIJ8ApYH1SqlkpVS02YEKw68vEL8M\nrOP2i4jLrLnUf/Uk0B9o++vfbfKvR7KimJMjdiGEsDJyxC6EEFZGil0IIayMFLsQQlgZKXYhhLAy\nUuxCCGFlpNiFEMLKSLELIYSVkWIXQggr8/8ASlABkCHQhtEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1110cdeb8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = gluon.loss.SoftMargin()\n",
    "\n",
    "with ag.record():    # start recording\n",
    "    theloss = loss(output, thelabel)\n",
    "theloss.backward()   # and compute the gradient\n",
    "\n",
    "plt.plot(output.asnumpy(), theloss.asnumpy(), label='Soft margin')\n",
    "plt.plot(output.asnumpy(), output.grad.asnumpy(), label='Gradient')\n",
    "# now compute the loss for y=-1\n",
    "theloss = loss(output, -thelabel)\n",
    "plt.plot(output.asnumpy(), theloss.asnumpy(), label='Soft margin for y=-1')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exponential Loss\n",
    "\n",
    "In some cases we *really* want to ensure that things are classified correctly we might replace $\\log(1 + \\exp(-yf))$ for its exponential counterpart, i.e. $\\exp(-yf)$. For instance, AdaBoost can be proven to minimize this loss function when it progressively weighs incorrectly classified data in an exponential way (as an aside, for two loss functions $l_1$ and $l_2$, the gradient $\\partial_w l_1(x,f(x))$ and $c \\partial_w l_2(x, f(x))$ are identical if $l_1 = c l_2$, hence changing the loss function or reweighting the data are equivalent). No matter, the loss function is available in 'MxNet Gluon' and it implements\n",
    "\n",
    "$$l(y, f) = \\exp(-y f)$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAD8CAYAAACVZ8iyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4VPXd///nOztLwhrWsCqLkA2ILKIsIrghuJS6oEBr\ny5e7uLR3a6XLj9vlbtXe3mpVKjfuVRSsCqKgVqgCVhTDviM7YQ1gIBBCts/vjxligAQCmclJMq/H\ndc01Z/mcc94TuOY1Z/scc84hIiKhLczrAkRExHsKAxERURiIiIjCQEREUBiIiAgKAxERQWEgIiIo\nDEREBIWBiIgAEV4XUF6NGzd2bdu29boMEZFqY8mSJQecc/HlaVttwqBt27akp6d7XYaISLVhZtvL\n21aHiURERGEgIiIKAxERoRqdMxCpzvLz88nIyCA3N9frUqQGiomJISEhgcjIyAteR0DCwMxeAYYC\n+51zif5pDwE/BzL9zX7vnJvjn/c74G6gELjPOfdpIOoQqaoyMjKIjY2lbdu2mJnX5UgN4pzj4MGD\nZGRk0K5duwteT6AOE70GXFPK9Kedc6n+18kg6ALcBnT1L/M3MwsPUB0iVVJubi6NGjVSEEjAmRmN\nGjWq8F5nQMLAObcAOFTO5sOBac65E865rcAmoGcg6hCpyhQEEiyB+L8V7BPI95jZSjN7xcwa+Ke1\nBHaWaJPhn3YGMxtrZulmlp6ZmVlak7PKzS9kyoLN/HvTgfNeVkQklAQzDF4ALgJSgT3A//qnlxZh\npT6I2Tk3xTmX5pxLi48v1010p4gMD2PKgi289c2O815WpKapW7duhdexe/dufvSjH5U5Pysri7/9\n7W/lbn+6MWPG8O6771aoRrkwQQsD59w+51yhc64IeJEfDgVlAK1KNE0AdgejhvAwY0jXZny+YT+5\n+YXB2IRISGnRosVZv6xPD4NztZeqI2hhYGbNS4zeBKz2D88CbjOzaDNrB3QAFgerjmu6NiMnr5AF\nG8//MJNITbd9+3YGDRpEcnIygwYNYscO31705s2b6d27N5deeikTJ04s3qvYtm0biYmJAKxZs4ae\nPXuSmppKcnIy3333HRMmTGDz5s2kpqbywAMPnNK+sLCQ3/zmNyQlJZGcnMxzzz131trmzZtHt27d\nSEpK4qc//SknTpwAYMKECXTp0oXk5GR+85vfAPCPf/yDxMREUlJS6NevX1D+VjVdoC4tfRsYADQ2\nswzgv4ABZpaK7xDQNuD/ATjn1pjZO8BaoAAY75wL2s/2Phc1ol6tSD5Zs5chXZsFazMi5fbwh2tY\nu/tIQNfZpUUc/3VD1/Ne7p577mHUqFGMHj2aV155hfvuu4+ZM2dy//33c//993P77bczefLkUped\nPHky999/PyNHjiQvL4/CwkIef/xxVq9ezfLlywFfeJw0ZcoUtm7dyrJly4iIiODQobKvOcnNzWXM\nmDHMmzePjh07MmrUKF544QVGjRrFjBkzWL9+PWZGVlYWAI888giffvopLVu2LJ4m5ydQVxPd7pxr\n7pyLdM4lOOdeds7d5ZxLcs4lO+eGOef2lGj/J+fcRc65Ts65jwNRQ1kiw8O46pKmzF27j7yComBu\nSqTaWbRoEXfccQcAd911F19++WXx9BEjRgAUzz9dnz59+POf/8wTTzzB9u3bqVWr1lm3NXfuXMaN\nG0dEhO83aMOGDctsu2HDBtq1a0fHjh0BGD16NAsWLCAuLo6YmBh+9rOf8f7771O7dm0A+vbty5gx\nY3jxxRcpLNQh4QsREncgX5PYjPeWZvD1loP063j+J6JFAulCfsFXlvO5RPGOO+6gV69ezJ49m6uv\nvpqXXnqJ9u3bl9neOVfu9TtX6jUlREREsHjxYubNm8e0adN4/vnn+de//sXkyZP55ptvmD17Nqmp\nqSxfvpxGjRqV+7NIiPRNdEWHxtSOCufj1Xu9LkWkSrnsssuYNm0aAFOnTuXyyy8HoHfv3rz33nsA\nxfNPt2XLFtq3b899993HsGHDWLlyJbGxsWRnZ5fafsiQIUyePJmCggKAsx4m6ty5M9u2bWPTpk0A\nvPHGG/Tv35+jR49y+PBhrrvuOp555pniw1GbN2+mV69ePPLIIzRu3JidO3eWuW4pXUiEQUxkOAM7\nN+GztXspLCr9F4dITZeTk0NCQkLx66mnnuLZZ5/l1VdfJTk5mTfeeIO//vWvADzzzDM89dRT9OzZ\nkz179lCvXr0z1jd9+nQSExNJTU1l/fr1jBo1ikaNGtG3b18SExN54IEHTmn/s5/9jNatW5OcnExK\nSgpvvfVWmbXGxMTw6quvMmLECJKSkggLC2PcuHFkZ2czdOhQkpOT6d+/P08//TQADzzwAElJSSQm\nJtKvXz9SUlIC+JcLDVbW7lhVk5aW5irycJuPVu7mnreWMW1sb3q31+6jVK5169ZxySWXeF1GueXk\n5FCrVi3MjGnTpvH222/zwQcfeF2WnEVp/8fMbIlzLq08y4fEOQOAKzs3oVZkOLNW7FYYiJzDkiVL\nuOeee3DOUb9+fV555RWvS5IgC5kwqB0VwZCuTZmzag8P3dCVqIiQOEImckGuuOIKVqxY4XUZUolC\n6htxeGoLsnLyWfidbkATESkppMLgig7xNKgdyczlQen9QkSk2gqpMIgMD+P65OZ8tnYvx04UeF2O\niEiVEVJhADA8tSW5+UV8tnaf16WIiFQZIRcGPVo3oGX9WsxcvsvrUkQq1b59+7jjjjto3749PXr0\noE+fPsyYMaNC63zooYd48sknAZg4cSJz5869oPUsX76cOXPmnPdyt99+O8nJycX3G5RW1/lKT0/n\nvvvuK3P+tm3bTrlH4lztTzdgwAAqcpl8sITM1UQnhYUZw1JbMGXBFjKzTxAfG+11SSJB55zjxhtv\nZPTo0cVfZNu3b2fWrFlntC0oKCjuP+h8PPLIIxdc3/Lly0lPT+e6664r9zJ79+7lq6++Yvv27Re8\n3dKkpaWRllb2pfknw+Bkn03nal9dhNyeAcAt3RMoLHK8vzTD61JEKsW//vUvoqKiGDduXPG0Nm3a\ncO+99wLw2muvMWLECG644QaGDBnC0aNHGTRoEN27dycpKemUG87+9Kc/0alTJ6666io2bNhQPL3k\ng2mWLFlC//796dGjB1dffTV79vj6qRwwYAAPPvggPXv2pGPHjixcuJC8vDwmTpzI9OnTSU1NZfr0\n6afUnpuby09+8hOSkpLo1q0bn3/+OeDr3mL//v2kpqaycOHCMj/78uXL6d27N8nJydx00018//33\nAHz77bckJyfTp08fHnjggeKutr/44guGDh0KwPz580lNTSU1NZVu3bqRnZ3NhAkTWLhwIampqTz9\n9NOntD969GhxrcnJycVdepTl7bffLr5z+sEHHwR8XX2PGTOGxMREkpKSivd6nn322eKuu2+77baz\nrvdChNyeAcDFTeqS1qYB09N3MrZfez2bVirXxxNg76rArrNZElz7eJmz16xZQ/fu3c+6ikWLFrFy\n5UoaNmxIQUEBM2bMIC4ujgMHDtC7d2+GDRvG0qVLmTZtGsuWLaOgoIDu3bvTo0ePU9aTn5/Pvffe\nywcffEB8fDzTp0/nD3/4Q/GNawUFBSxevJg5c+bw8MMPM3fuXB555BHS09N5/vnnz6hr0qRJAKxa\ntYr169czZMgQNm7cyKxZsxg6dGhx/0RlGTVqFM899xz9+/dn4sSJPPzwwzzzzDP85Cc/YcqUKVx2\n2WVMmDCh1GWffPJJJk2aRN++fTl69CgxMTE8/vjjPPnkk3z00UeALzxOevTRR6lXrx6rVvn+fU8G\nT2l2797Ngw8+yJIlS2jQoAFDhgxh5syZtGrVil27drF6te8RMCe75H788cfZunUr0dHRQemmOyT3\nDAB+nNaKLZnHWLK97H8skZpq/PjxpKSkcOmllxZPGzx4cHG30s45fv/735OcnMxVV13Frl272Ldv\nHwsXLuSmm26idu3axMXFMWzYsDPWvWHDBlavXs3gwYNJTU3lv//7v8nI+GEv/OabbwagR48epzzv\noCxffvkld911F+DrwK5NmzZs3LixXJ/z8OHDZGVl0b9/f+CHrrCzsrLIzs7msssuA8ruprtv3778\n53/+J88++yxZWVnnPHw2d+5cxo8fXzzeoEGDMtt+++23DBgwgPj4eCIiIhg5ciQLFiygffv2bNmy\nhXvvvZdPPvmEuLg4AJKTkxk5ciRvvvnmBR3GO5eQ3DMAuD65OQ9/uIZ30neS1rbsftVFAu4sv+CD\npWvXrqccspg0aRIHDhw45Vh3nTp1ioenTp1KZmYmS5YsITIykrZt25Kbmwucu5tr5xxdu3Zl0aJF\npc6PjvadpwsPDy/uwfRc6wu08q5zwoQJXH/99cyZM4fevXuf8wR5ILrpbtCgAStWrODTTz9l0qRJ\nvPPOO7zyyivMnj2bBQsWMGvWLB599FHWrFkT0FAI2T2DOtERDE1uwUcr93BU9xxIDXfllVeSm5vL\nCy+8UDwtJyenzPaHDx+mSZMmREZG8vnnnxefpO3Xrx8zZszg+PHjZGdn8+GHH56xbKdOncjMzCwO\ng/z8fNasWXPW+s7W9XW/fv2YOnUqABs3bmTHjh106tTp7B/Yr169ejRo0KD4nMLJrrAbNGhAbGws\nX3/9NVB2N92bN28mKSmJBx98kLS0NNavX3/ObrpLHuo622GiXr16MX/+fA4cOEBhYSFvv/02/fv3\n58CBAxQVFXHLLbfw6KOPsnTpUoqKiti5cycDBw7kL3/5C1lZWRw9erRcf4PyCtkwAPjxpa3IyStk\n9krdkSw1m5kxc+ZM5s+fT7t27ejZsyejR4/miSeeKLX9yJEjSU9PJy0tjalTp9K5c2cAunfvzq23\n3kpqaiq33HILV1xxxRnLRkVF8e677/Lggw+SkpJCamoqX3311VnrGzhwIGvXri31BPIvfvELCgsL\nSUpK4tZbb+W1114r3rsoj9dff50HHniA5ORkli9fzsSJEwF4+eWXGTt2LH369ME5V2o33c8880zx\ns5Vr1arFtddeS3JyMhEREaSkpJxxSesf//hHvv/+++JlTp7sLk3z5s157LHHGDhwICkpKXTv3p3h\nw4eza9cuBgwYQGpqKmPGjOGxxx6jsLCQO++8s/gk+q9+9Svq169f7r9BeYRMF9alcc4x+OkF1I2O\nYOb4vgFdt0hJ1a0L61Bw9OhR6tatC/hOzu7Zs6f4eQ7VUUW7sA7pPQMzY2Sv1izfmcXKDD1EWySU\nnHxEZmJiIgsXLuSPf/yj1yV5KqTDAOCWHgnUjgrn74sCe+OKiFRtt956K8uXL2f16tXMnj2b+PjQ\nfj56yIdBXEwkN3dvyawVuzl0LM/rcqQGqy6HZKX6CcT/rZAPA4BRfdqSV1DEtG93eF2K1FAxMTEc\nPHhQgSAB55zj4MGDxMTEVGg9IXufQUkdm8bSp30jpn69g7FXtCciXBkpgZWQkEBGRgaZmXqwkgRe\nTEwMCQkJFVpHQMLAzF4BhgL7nXOJ/mkNgelAW2Ab8GPn3PfmuyPjr8B1QA4wxjm3NBB1VMToy9ow\n7s2lzF23n2sSm3ldjtQwkZGRtGvXzusyRMoUqJ/ArwHXnDZtAjDPOdcBmOcfB7gW6OB/jQVeoAq4\n6pKmtKxfi5e/3OJ1KSIilS4gYeCcWwAcOm3ycOB1//DrwI0lpv/d+XwN1Dez5oGooyIiwsO4+/J2\nfLvte5ZsP/2jiIjUbME8ON7UObcHwP/exD+9JbCzRLsM/7QzmNlYM0s3s/TKONZ6W89W1K8dyf/N\n196BiIQWL86UltaLU6mXWDjnpjjn0pxzaZVxDXDtqAhG9W7DZ+v2sWl/YPv9EBGpyoIZBvtOHv7x\nv+/3T88AWpVolwBUmc6BRl3WlqjwMF5coL0DEQkdwQyDWcBo//Bo4IMS00eZT2/g8MnDSVVB47rR\njEhLYMayXew7kut1OSIilSIgYWBmbwOLgE5mlmFmdwOPA4PN7DtgsH8cYA6wBdgEvAj8IhA1BNLY\nKy6i0Dle+GKz16WIiFSKgNxn4Jy7vYxZg0pp64DxpbStMlo3qs0t3Vvy1uIdjOt/Ec3qVezOPhGR\nqk632pbh3is7UFTk+NsXm7wuRUQk6BQGZWjVsDYj0hKYtngnu7OOe12OiEhQKQzOYvzAi3E4Jn2u\nvQMRqdkUBmeR0KA2P05rxTvpO9l+8JjX5YiIBI3C4BzuH9SBiLAw/vLJBq9LEREJGoXBOTSJi2Fs\nv/bMXrWHJdu/97ocEZGgUBiUw9h+7YmPjebPc9bp4SQiUiMpDMqhTnQEvx7ckSXbv+eT1Xu9LkdE\nJOAUBuU0Iq0VnZrG8ueP15GbX+h1OSIiAaUwKKfwMOOhYV3Zeeg4f1M3FSJSwygMzkOfixoxPLUF\nk+dvZtsBXWoqIjWHwuA8/eG6S4gKD+O/Zq3RyWQRqTEUBuepSVwMvxrckfkbM/lYJ5NFpIZQGFyA\n0X3a0LVFHBM/WMP3x/K8LkdEpMIUBhcgIjyM//lRClk5eTz84RqvyxERqTCFwQXq0iKO8QMvZuby\n3Xy2dp/X5YiIVIjCoALGD7yYzs1i+f2MVWTl6HCRiFRfCoMKiIoI48kRvsNFD763UlcXiUi1pTCo\noMSW9fjt1Z35dM0+pn6zw+tyREQuiMIgAO6+vB39O8bz6Edr2bA32+tyRETOm8IgAMLCjCdHpBAb\nE8k9by3leJ76LhKR6kVhECDxsdE89eMUNmUe5Xfv6/yBiFQvCoMA6tcxnl8P7sjM5bt5+cutXpcj\nIlJuCoMAGz/wYq5NbMaf56zjy+8OeF2OiEi5BD0MzGybma0ys+Vmlu6f1tDMPjOz7/zvDYJdR2Ux\n850/6NAklvFvLVXvpiJSLVTWnsFA51yqcy7NPz4BmOec6wDM84/XGHWiI5gyqgdhBqNfXcyBoye8\nLklE5Ky8Okw0HHjdP/w6cKNHdQRNm0Z1eHnMpew7ksvdr31LTl6B1yWJiJSpMsLAAf80syVmNtY/\nralzbg+A/71JJdRR6bq3bsBzt3dn1a7DjJ+6lILCIq9LEhEpVWWEQV/nXHfgWmC8mfUr74JmNtbM\n0s0sPTMzM3gVBtHgLk159MZEPt+Qya//sYLCIl1yKiJVT0SwN+Cc2+1/329mM4CewD4za+6c22Nm\nzYH9ZSw7BZgCkJaWVm2/RUf2asPh4/n85ZMNhJvxPyNSCA8zr8sSESkW1D0DM6tjZrEnh4EhwGpg\nFjDa32w08EEw66gKfjHgYn49uCPvL9vFhPdWUqQ9BBGpQoK9Z9AUmGFmJ7f1lnPuEzP7FnjHzO4G\ndgAjglxHlXDvoA4UFDn+Ou87Coocf/lRMpHhutVDRLwX1DBwzm0BUkqZfhAYFMxtV1W/vKoDkeHG\nk//cyOHj+Uy6ozu1osK9LktEQpx+llYyM+OeKzvw3zcm8vmG/Yx65RsO5+R7XZaIhDiFgUfu7N2G\n527vxvKdWdz0wr/ZqjuVRcRDCgMPDU1uwZt39yIrJ58bJ/1bfRmJiGcUBh7r1b4RH4zvS9O4aEa/\nupjX/r1V3V+LSKVTGFQBrRrW5r3/uIyBneJ56MO1/GLqUg4f13kEEak8CoMqIjYmkil3pfH76zrz\n2dp9XP/sQpbt+N7rskQkRCgMqpCwMGNsv4t4Z1wfnIMRkxfx1GcbyStQn0YiElwKgyqoe+sGzLnv\nCm5IacGz875j2PNfsnrXYa/LEpEaTGFQRdWrHcnTt6by4qg0Dh3LY/ikf/PEJ+vVFbaIBIXCoIob\n3KUpn/2qPzd1a8kLX2xm0P/OZ9aK3briSEQCSmFQDdSrHcmTI1J4d1wfGtaJ4r63l3HrlK9ZlaFD\nRyISGAqDaiStbUNm3XM5f74pie/2ZXPD818y7o0lbNib7XVpIlLNBf15BhJY4WHGHb1ac0NKc17+\ncisvLdzKp2v3MiylBfdeeTEXN4n1ukQRqYasuhx7TktLc+np6V6XUeV8fyyP/1uwhde+2kpufhGD\nOjfh5/3a06tdQ/xdh4tIiDKzJc65tHK1VRjUDAePnuCNr7fz90XbOXQsj5SEeozq05brk5sTE6ku\nskVCkcIghOXmF/Le0gxe/nIrWzKPERcTwU3dWnJbz9Zc0jzO6/JEpBIpDATnHN9sPcTbi3fw8aq9\n5BUWkdgyjmEpLbg+uQUt69fyukQRCTKFgZzi0LE83l+awawVu1npvxw1rU0DbkhpwVVdmioYRGoo\nhYGUaduBY3y0cjcfrtjDhn2+S1I7N4tlYOcmXNm5Cd1a1SdCz2UWqREUBlIum/Yf5fP1+5m3fh/p\n276noMhRr1Ykvdo1pHf7RvRu34jOzWIJC9NVSSLV0fmEge4zCGEXN6nLxU3q8vN+7Tl8PJ+F32Wy\nYGMmX285xD/X7gOgXq1IerZrSLfW9UlNqE9iQj3iYiI9rlxEAk1hIIDvS39ocguGJrcAYFfWcb7Z\ncpBvthzim60H+cwfDgAXxdchJaE+SQn16NQslk5NY2lUN9qr0kUkAHSYSMolKyePlRmHWbEzixUZ\nh1mRkUVm9oni+Y3qRNGxaSwdm9alY7NY2jWuQ5tGdWgeF6PDTCIe0WEiCbj6taPo1zGefh3jAd+l\nq5nZJ9i47ygb9mWzcW82G/Zl8+6SDI7lFRYvFxUeRkLDWrRtVIfWDWvTplFtWtSvRfN6MTSLi6FR\n3WjCFRYinvMsDMzsGuCvQDjwknPuca9qkfNnZjSJi6FJXAyXd2hcPL2oyLH78HG2H8xh28Fj7DiY\nw/aDOWw/lMM3Ww6eEhTg62upaWw0TevF0LxeDE3jYmhcN5qGdaJoWCeKRsXv0cTVilAXGyJB4kkY\nmFk4MAkYDGQA35rZLOfcWi/qkcAJCzMSGtQmoUFt+l7c+JR5zjkOHstjT1Yue4/ksvfwcfYeyWXP\n4Vz2Hs5l/d5svtiQSc5pgXFSRJjRwB8QcTGRxMZEEBsTQVytk8ORp0yPjYmkbnQEtSLDqRXlf0WG\na09EpBRe7Rn0BDY557YAmNk0YDigMKjBzIzGdaNpXDeaJOqV2e54XiGHcvI4dDSPg8dOcOhYHoeO\n5XHwmG/aoZw8snPz2Xskl43788nOLSA7t4DCovKd/4qKCKO2PxhOBkTtqHBiIn3DURFhvlf4D++R\np42fbBNZYlp0RBgR4Ua4GeFh5hsOCztlPMyMiLAS80/OCwsj/JRx07kWqVRehUFLYGeJ8Qygl0e1\nSBVTKyqcllG1zuvOaOccx/MLOXK8gOzcfI7k+t5z8grJySvkeH4hx/MKOJ5XRE5+AcfzCjmeV0hO\nfiG5/jbZuQVkZp8gr6CIEwVF5BcWkVdYRJ5/OL+w9LAxioiigEgKiKCQCIoIo4gICgm3U8cjKCKc\nQsL97xFWoq3/3Tfumx8V5gijCDMIN0c4jjB8w2Ynh4sIwxFm+N5xhJ1saw7ztw+jyNfGOcKsCAPC\n/OvwLeffjn8YM8C3/A/vJz8zGK54mJPjBuYcYMXjnNK2lGVKjJ8xrXh9JdZfsq07Oe309ZRed8n1\nX4iKxPOFbrcwMpZev5xagS2Xj1dhUNrf9Iy/lJmNBcYCtG7dOtg1STVmZtSOiqB2VATNYiPhxBHI\nPQ55xyA/x/fKy/lhOL+0ecd9w4V5UHDC9148nI8rPAEFJ3CFeVCQB4UnsMJ8rCjf2w/vTnsvJ1+E\n4IsPMxx2MhIo+dXlSnx1F2/SDBy44kmnfvWeuWzJNmeu9/TST/8q/2F+aV/7pS/zw7zyRER5uQos\ne2Hbzckvey86kLwKgwygVYnxBGD36Y2cc1OAKeC7tLRySpMqI+8YHDsAOQfg2EH/+wE4fghyD0Pu\nEd/7iSO+4RP+8byj57ediFoQVRsia0NkLQiPhogoCI+CiBiIqQfhUVh4FEREY+GR/jbRUDzsbx8e\nBRYGYRElXuH+VwRYeOnTT46fPt/CTnvZD8PYadOtjOmnt/e1VacjUpJXYfAt0MHM2gG7gNuAOzyq\nRSpb3jE4sgeO7IJs//uR3b5p2bt9X/jHDkDB8dKXD4v0fUHHxEF0nG+4cROIrnfa9DiIquv7ko/y\nf9FH1vG9R/nfI2pBmL4WRTwJA+dcgZndA3yK79LSV5xza7yoRYKgqMj3Jf/9Vji09cz33Kwzl4mp\nB3EtIbY5NO4EdRpD7Ub+98anjkfH+Y9ni0igeHafgXNuDjDHq+1LADjn+0W/fx3sX+t7z1wHmRt8\nx95PsnCo3xoatoPE7lAvwffFH9ccYlv43qPqePc5RER3IEs5OQeHM2DXEti9FHYthT0r4cThH9rU\nbQZNOkP30dD4YmjQzhcA9Vr5jq2LSJWlMJDSFRbA3hWw7UvYvgh2pcOxTN+88ChomghJP4KmXSD+\nEmhyCdRu6G3NInLBFAbi4xzsWwOb5/0QAHm+h9/QqANcPBhadve9mib6rqQRkRpDYRDK8nJg20LY\n+Als/CccyfBNb9wRkkdA28uhzeUQ29TbOkUk6BQGoSb/OHz3T1j9ni8ACo77Lre8aCD0/y10GAxx\nLbyuUkQqmcIgFBQVwubPYdU7sH6276asOvHQbSR0us63B6DDPiIhTWFQkx3ZDcvehKV/h8M7fdfy\nd70JEm+BtldAuP75RcRH3wY1jXOw/d+w6G+w8WNwRdB+AAx+BDpfrz0AESmVwqCmKCyAdR/AV8/B\n7mW+u3b7/hK63wUN23tdnYhUcQqD6q6wAFZOh/lPQNZ2aHgRDH0aUm739b0jIlIOCoPqqqgI1s6A\nzx+Dg99B8xS4+s++E8LqeE1EzpPCoDra9iV8MgH2rvLd/Xvrm9B5qDpvE5ELpjCoTg5nwD//P1jz\nvq+/n5tf9F0ZFBbudWUiUs0pDKqDwgJY9BzM/4vv6qD+E6Dv/b4++kVEAkBhUNXtWwsf/MJ3hVDn\nob7zAg3aeF2ViNQwCoOqqrAA/v00fPGE74ldI17z3TAmIhIECoOq6HAGvHs37PzaFwDXPel7wpeI\nSJAoDKqaDR/DzP+Awny4+SVf76EiIkGmMKgqigph7kPw1bPQLNl3WKjRRV5XJSIhQmFQFeQehvd+\n5utaOu0TZzP2AAALkElEQVRuuOYx9SEkIpVKYeC1Q1vgrdvg0GZfNxJpP/W6IhEJQQoDL2Wkw9QR\ngIO7ZkC7fl5XJCIhSmHglU3zYPpdUDce7nxf5wdExFPq0cwLq9+Ht271dS39038qCETEcwqDyrZi\nOrz7U0hIgzEf6WHzIlIlBC0MzOwhM9tlZsv9r+tKzPudmW0ysw1mdnWwaqhyVr0LM8f5njl85/tQ\nq77XFYmIAME/Z/C0c+7JkhPMrAtwG9AVaAHMNbOOzrnCINfirTUz4P2x0LoP3DFdncyJSJXixWGi\n4cA059wJ59xWYBPQ04M6Ks93c333ESRc6g+COl5XJCJyimCHwT1mttLMXjGzBv5pLYGdJdpk+Ked\nwczGmlm6maVnZmYGudQg2b0M3hkFTS6Bkf+A6FivKxIROUOFwsDM5prZ6lJew4EXgIuAVGAP8L8n\nFytlVa609Tvnpjjn0pxzafHx8RUp1RuHtvruI6jdCEa+6+t9VESkCqrQOQPn3FXlaWdmLwIf+Ucz\ngFYlZicAuytSR5WUcwjevAWKCuDO9yC2mdcViYiUKZhXEzUvMXoTsNo/PAu4zcyizawd0AFYHKw6\nPFFY4Lt89PBOuH0axHf0uiIRkbMK5tVEfzGzVHyHgLYB/w/AObfGzN4B1gIFwPgadyXR3P+CLZ/D\n8EnQurfX1YiInFPQwsA5d9dZ5v0J+FOwtu2ple/Aoueh51jodqfX1YiIlIvuQA6kvath1r3Qpq/v\nWcUiItWEwiBQ8o7Buz+BmHow4nUIj/S6IhGRclOvpYHyyQQ48B2MmunriVREpBrRnkEgrH4flv4d\nLv8VtB/gdTUiIudNYVBRWTvgw1/6upoY+HuvqxERuSAKg4pwDmbdB64QbnlJ5wlEpNrSOYOKWPp3\n3/0E1z8FDdp6XY2IyAXTnsGFytoJn/4B2l4BPX7idTUiIhWiMLgQzsFHvwRXBMOfhzD9GUWketNh\nogux5n3YNBeueUKHh0SkRtBP2vN1Itt3eKh5CvT8udfViIgEhPYMztcXj0P2Xrj1TQgL97oaEZGA\n0J7B+di3Fr5+AbqPgoQ0r6sREQkYhUF5OQcf/9bX99BVD3ldjYhIQCkMymvDx7Btoe8u49oNva5G\nRCSgFAblUZgPn02ERh2gxxivqxERCTidQC6Ppa/Dwe/gtrfV5YSI1EjaMziX3CPw+WO+B9Z0utbr\nakREgkJ7Bufy1XOQcwCGvANmXlcjIhIU2jM4m5xDvktJLxkGLXt4XY2ISNAoDM5m0fOQdxQG/M7r\nSkREgkphUJZjB+Gb/4OuN0LTLl5XIyISVAqDsnz1rO8h9/0neF2JiEjQKQxKc+wALH4REm+BJp29\nrkZEJOgqFAZmNsLM1phZkZmlnTbvd2a2ycw2mNnVJaZf45+2ycyq5s/ubyZDfg70/63XlYiIVIqK\n7hmsBm4GFpScaGZdgNuArsA1wN/MLNzMwoFJwLVAF+B2f9uq48RR315B5+shvpPX1YiIVIoK3Wfg\nnFsHYGdefz8cmOacOwFsNbNNQE//vE3OuS3+5ab5266tSB0BtewNyM2Cvr/0uhIRkUoTrHMGLYGd\nJcYz/NPKml41FObDoknQ+jJodanX1YiIVJpz7hmY2VygWSmz/uCc+6CsxUqZ5ig9fNxZtj0WGAvQ\nunXrc1QaAKvfh8M74bong78tEZEq5Jxh4Jy76gLWmwG0KjGeAOz2D5c1vbRtTwGmAKSlpZUZGgHh\nnO9y0vjO0GFIUDclIlLVBOsw0SzgNjOLNrN2QAdgMfAt0MHM2plZFL6TzLOCVMP52fYl7FsNfcZD\nmK64FZHQUqETyGZ2E/AcEA/MNrPlzrmrnXNrzOwdfCeGC4DxzrlC/zL3AJ8C4cArzrk1FfoEgbJ4\nCtRqAEkjvK5ERKTSVfRqohnAjDLm/Qn4UynT5wBzKrLdgDucAetnw2X3QGQtr6sREal0Oh4CkP4q\nuCJIu9vrSkREPKEwKDgBS17zPbimQRuvqxER8YTCYM1M38Nrev7c60pERDyjMPj2Jd+D7tsP9LoS\nERHPhHYY7F8PGYuhx2g90lJEQlpoh8GyNyAsApJv87oSERFPhW4YFOTBimm+E8d1472uRkTEU6Eb\nBhs/8Z047jbK60pERDwXumGw7A2IbQ4XXel1JSIingvNMDiyGzbNhdQ7ILxCN2GLiNQIoRkGK6b5\n7jjudqfXlYiIVAmhGQar/gGtekPD9l5XIiJSJYReGOxdDfvXQtKPvK5ERKTKCL0wWPUP370FXW/2\nuhIRkSojtMKgqAhWv+e7gqhOI6+rERGpMkIrDHZ+7XvGsR5gIyJyitAKg1X/gMja0Ok6rysREalS\nQicMCvJgzQxfEETX9boaEZEqJXTCYOt8OP69riISESlF6ITB2g8gKlbdT4iIlCI0wqCwADbMgY5X\nQ0S019WIiFQ5oREGO76CnIPQZZjXlYiIVEmhEQbrPoSIWnDxVV5XIiJSJdX8MCgq8oXBxYMgqo7X\n1YiIVEkVCgMzG2Fma8ysyMzSSkxva2bHzWy5/zW5xLweZrbKzDaZ2bNmQX748K50yN4Dl+gQkYhI\nWSq6Z7AauBlYUMq8zc65VP9rXInpLwBjgQ7+1zUVrOHs1s2CsEjfyWMRESlVhcLAObfOObehvO3N\nrDkQ55xb5JxzwN+BGytSwzkKhLWzoH1/qFU/aJsREanugnnOoJ2ZLTOz+WZ2hX9aSyCjRJsM/7Tg\nyM+Bdv0g5fagbUJEpCY45zMfzWwu0KyUWX9wzn1QxmJ7gNbOuYNm1gOYaWZdgdLOD7izbHssvkNK\ntG7d+lylnimqDgx//vyXExEJMecMA+fceV+P6Zw7AZzwDy8xs81AR3x7AgklmiYAu8+yninAFIC0\ntLQyQ0NERComKIeJzCzezML9w+3xnSje4pzbA2SbWW//VUSjgLL2LkREpJJU9NLSm8wsA+gDzDaz\nT/2z+gErzWwF8C4wzjl3yD/vP4CXgE3AZuDjitQgIiIVZ76Leqq+tLQ0l56e7nUZIiLVhpktcc6l\nnbtlKNyBLCIi56QwEBERhYGIiCgMRESEanQC2cwyge1e13GeGgMHvC6ikukzhwZ95uqhjXMuvjwN\nq00YVEdmll7eM/k1hT5zaNBnrnl0mEhERBQGIiKiMAi2KV4X4AF95tCgz1zD6JyBiIhoz0BERBQG\nlcbMfmNmzswae11LsJnZ/5jZejNbaWYzzKzGPmbOzK4xsw3+Z3pP8LqeYDOzVmb2uZmt8z///H6v\na6oMZhbuf1jXR17XEiwKg0pgZq2AwcAOr2upJJ8Bic65ZGAj8DuP6wkKfzftk4BrgS7A7WbWxduq\ngq4A+LVz7hKgNzA+BD4zwP3AOq+LCCaFQeV4GvgtZ3mqW03inPunc67AP/o1pz7QqCbpCWxyzm1x\nzuUB04DhHtcUVM65Pc65pf7hbHxfkMF7dG0VYGYJwPX4ut6vsRQGQWZmw4BdzrkVXtfikZ9Sc59Z\n0RLYWWI8uM/0rmLMrC3QDfjG20qC7hl8P+aKvC4kmM752Es5t7M9Jxr4PTCkcisKvvI8G9vM/oDv\nsMLUyqytEp3XM71rEjOrC7wH/NI5d8TreoLFzIYC+/2P7x3gdT3BpDAIgLKeE21mSUA7YIXvKZ8k\nAEvNrKdzbm8llhhw53o2tpmNBoYCg1zNvX45A2hVYvysz/SuKcwsEl8QTHXOve91PUHWFxhmZtcB\nMUCcmb3pnLvT47oCTvcZVCIz2wakOeeqW2dX58XMrgGeAvo75zK9ridYzCwC3wnyQcAu4FvgDufc\nGk8LCyL/s8tfBw45537pdT2Vyb9n8Bvn3FCvawkGnTOQYHgeiAU+M7PlZjbZ64KCwX+S/B7gU3wn\nUt+pyUHg1xe4C7jS/2+73P+rWao57RmIiIj2DERERGEgIiIoDEREBIWBiIigMBARERQGIiKCwkBE\nRFAYiIgI8P8Dv7KBk/vwe58AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x111014a20>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = gluon.loss.Exponential()\n",
    "\n",
    "# getting data ready\n",
    "thelabel = nd.ones_like(output)\n",
    "with ag.record():    # start recording\n",
    "    theloss = loss(output, thelabel)\n",
    "theloss.backward()   # and compute the gradient\n",
    "\n",
    "plt.plot(output.asnumpy(), theloss.asnumpy(), label='Logistic loss')\n",
    "plt.plot(output.asnumpy(), output.grad.asnumpy(), label='Gradient of logistic loss')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Langford's VW loss\n",
    "\n",
    "One of the more unusual loss functions is John Langford's VW style loss. It is essentially a cut variant of Huber's robust loss, and it works by piecing together a linear, quadratic and constant part of a loss function. The benefit of this choice is that its gradient is bounded for significant misclassification, that its gradient vanishes for highly confident classification and that there is a graduation in terms of how poorly classified data is. We have\n",
    "\n",
    "$$l(y,f) = \\begin{cases}\n",
    "  0 & \\text{ if } 1 < y f \\\\\n",
    "  \\frac{1}{2} f^2 & \\text { if } 0 \\leq yf \\leq 1 \\\\\n",
    "  \\frac{1}{2}-yf & \\text{ otherwise}\n",
    "\\end{cases}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8jvX/wPHXx5BKyCKKGeU822xz2nKqUJKZQ4wck/qS\npHJKkQ6U+uYQFSWqn6IDOolyHCHGRg4hozKnUfkSZrvv9++Pe1ubDTvc23Uf3s/HwyO7d93X9b7X\nvHxcu+77NiKCUkopz1HM6gGUUko5l4ZdKaU8jIZdKaU8jIZdKaU8jIZdKaU8jIZdKaU8jIZdKaU8\njIZdKaU8jIZdKaU8THErDnrTTTeJv7+/FYdWSim3tXXr1pMiUuFq21kSdn9/f2JjY604tFJKuS1j\nzG+52U5PxSillIfRsCullIfRsCullIex5By7skZKSgqHDx/mwoULVo+i3FipUqWoUqUKJUqUsHoU\ndRkadi9y+PBhbrjhBvz9/THGWD2OckMiwqlTpzh8+DDVq1e3ehx1GXoqxotcuHABX19fjbrKN2MM\nvr6++q8+F6dh9zIadVVQ+j3k+twq7PF//M2stQesHkMppfJl45GNFMXbkbpV2BdtO8yk735h2or9\nVo+i8qFVq1YsX748y21Tp05l8ODBREVFsWTJkozba9euzUsvvZTxcZcuXVi0aFGujjNx4sSrbnPo\n0CECAgJyOblD6dKl87S9Upm9s/0dBv0wiOWHll994wJyq7CPv78+XUKqMGXFPqau2Gf1OCqPoqOj\nWbBgQZbbFixYQHR0NOHh4WzYsAGAU6dOUbp0aTZu3Jix3caNGwkPD8/VcXITdqWK0tvb32Zm/Ew6\n3taRNtXaFPrx3CrsPsUMk7sG0iWkClNX7Ne4u5muXbvyzTffkJycDDhWzUeOHOGOO+4gIiIiI+wb\nNmygQ4cOJCUlISIcPHiQa6+9lkqVKmXZ39GjR2nRogXBwcEEBASwbt06Ro8ezfnz5wkODqZXr148\n99xzTJs2LeM+Y8eOZfr06Vn2Y7PZGDFiBI0aNSIwMJBZs2Zd8XGICCNGjCAgIIAGDRqwcOHCy85j\ns9no169fxrZTpkwp8NdRuZe3t7/NW/Fv0fG2jrwQ/gI+xXwK/Zhud7ljetyNgakr9iMCw9vUsnos\ntzPh613sPvI/p+6z3i1lGH9//ct+3tfXl8aNG7Ns2TIiIyNZsGAB3bt3xxhDaGgoO3fu5OLFi2zY\nsIGWLVuSkJDAnj17iIuLIyIiItv+Pv74Y9q1a8fYsWOx2WycO3eO5s2bM2PGDOLj4wHHXx6dO3dm\n2LBh2O12FixYwObNmzlz5kzGfubMmUPZsmXZsmULycnJRERE0LZt28tezrdo0SLi4+PZvn07J0+e\npFGjRrRo0SLHeeLj40lMTGTnzp0A/P333wX5Eis383b827y1vWijDm4YdnDE/dUugRhg2krH+XaN\nu3tIPx2THvb3338fgGuuuYb69euzbds2Nm3axMiRI0lISGDDhg3ExcXleBqmUaNGDBgwgJSUFDp1\n6kRwcHC2bfz9/fH19SUuLo7jx4/TsGFDfH19s4T9+++/Z8eOHXz++ecAnD59mv3791827OvXryc6\nOhofHx9uvvlmWrZsyZYtW3Kcp0aNGiQkJDB06FDuu+8+2rZt64wvo3ID6VGPvC2SCeETiizq4KZh\nh3/jDhr3/LjSyrowderUiSeffJJt27Zx/vx5QkJCMj4XHh5OTEwMZ86c4cYbb6Rp06bMmDGDuLg4\nHn300Wz7atGiBTExMXz77bf07t2bESNG0KdPn2zbDRw4kHnz5nHs2DEGDBiQ7fMiwptvvkm7du1y\n9Rgud1XD5ebZvn07y5cvZ+bMmXz66acZf5kpz/VW/Fu8vf1tS6IOOL5Ji/pXaGioOIvNZpenP42X\naqO+kf9+v1fsdrvT9u1pdu/ebfUIIiLSrVs3CQoKkvHjx2e5ffHixVKjRg0ZMGCAiIikpKRI3bp1\npXz58mKz2bLt59ChQ5KSkiIiIlOmTJFhw4aJiEi5cuXk4sWLGdslJydLrVq1pHr16pKamioiIgcP\nHpT69euLiMisWbMkMjIy4z579+6Vs2fPZjve9ddfLyIiX3zxhbRt21ZSU1PlxIkT4ufnJ0ePHs1x\nnqSkJDl9+rSIiMTFxUlQUFD+vmguxlW+l1zRzLiZEjAvQJ5d/6yk2lKdum8gVnLRWLddsacrln5a\nxsD0lftBhOFtaumTKFxYdHQ0nTt3znaFTHh4OAkJCYwZMwaA4sWLU7FiRapWrUqxYtl/zr9mzRpe\ne+01SpQoQenSpfnwww8BGDRoEIGBgYSEhDB//nxKlixJ69atKVeuHD4+2VdOAwcO5NChQ4SEhCAi\nVKhQIcull5eKiopi48aNBAUFYYxh8uTJVKpUiQ8++CDbPImJifTv3x+73Q7ApEmT8v11U64vfaXe\n6fZOTAifQDFjzfUpRorgYvlLhYWFibPfaMNuF8Ys+pmFsX8w9M7beVLjns2ePXuoW7eu1WMUObvd\nTkhICJ999hk1a9a0ehyP4K3fS1dSFFE3xmwVkbCrbef2K/Z0xYoZJnVuAMCbq34F0Lgrdu/eTYcO\nHYiKitKoq0LjKiv1dB4Tdvg37sY44i4CT7XVuHuzevXqkZCQYPUYykOJCG9tf4t3tr/jMlEHDws7\nOOI+McoR9xmrf0UQnm5bW+OulHKqzFGPuj2K58Ofd4mog5PCbow5BJwBbEBqbs4BFaZixQwvd2oA\nGGaudrxomMZdKeUsIsLM+JnM2jHL5aIOzl2xtxaRk07cX4E44u54kaeZqw8gAiPaadyVUgWTOeqd\na3ZmfLPxLhV18MBTMZmlx90YeGvNAQQYqXFXSuWTiDAjfgazd8x22aiD814ETIDvjTFbjTGDctrA\nGDPIGBNrjIlNSkpy0mGvrlgxw0uRAfRq4sfbaw7w6rK9RfJ6yOryjh8/Ts+ePalRowahoaE0a9aM\nxYsX53t/zz//PK+//joA48aNY8WKFfnaT3x8PEuXLs33HMqzZY56l5pdXDbq4LwVe4SIHDHGVAR+\nMMb8IiIxmTcQkdnAbHBcx+6k4+ZKsWKGFyMdp2XeSXujjlH36MrdCiJCp06d6Nu3Lx9//DEAv/32\nG1999VWW7VJTUylePO/fni+88EK+Z4uPjyc2Npb27dvnex/KM4kIb8a9ybs/v0uXml0Y12ycy0Yd\nnLRiF5Ejaf89ASwGGjtjv86UHvcHm/rxztoDvLLsF125W2DVqlWULFkyy2u/VKtWjaFDhzJv3jy6\ndevG/fffT9u2bTl79ix33XUXISEhNGjQgC+//DLjPi+//DK1a9fm7rvvZu/evRm39+vXL+PFvLZu\n3UrLli0JDQ2lXbt2HD16FHC84ceoUaNo3LgxtWrVYt26dVy8eJFx48axcOFCgoODM16KVyl3izo4\nYcVujLkeKCYiZ9J+3xbI/7KpEGVeuc9amwACo++t450r9+9Gw7GfnbvPSg3g3leuuMmuXbuyvPDX\npTZu3MiOHTsoX748qampLF68mDJlynDy5EmaNm1Kx44d2bZtGwsWLCAuLo7U1FRCQkIIDQ3Nsp+U\nlBSGDh3Kl19+SYUKFVi4cCFjx47NeAGu1NRUNm/ezNKlS5kwYQIrVqzghRdeIDY2lhkzZhT8a6E8\ngjtGHZxzKuZmYHFaHIsDH4vIMifst1AY44i7wTArxvHEFa+NuwsYMmQI69evp2TJkgwZMoQ2bdpQ\nvnx5wPGH6plnniEmJoZixYqRmJjI8ePHWbduHVFRUVx33XUAdOzYMdt+9+7dy86dO2nTxvFuNTab\njcqVK2d8vnPnzgCEhoZy6NChQn6Uyh1ljnrXWl15rulzbhF1cELYRSQBCHLCLEXGGMMLkY6XrZ0V\nk4AAY7wt7ldZWReW+vXr88UXX2R8PHPmTE6ePElYmOOpD9dff33G5+bPn09SUhJbt26lRIkS+Pv7\nc+HCBYCr/r8SEerXr5/l7fUyu+aaawDw8fEhNTW1QI9JeR4RYXrcdN77+T23izq42VvjOVN63Ps0\nq8bsmAQmLt2j59yLwJ133smFCxd4++23M247d+5cjtuePn2aihUrUqJECVavXs1vv/0GOF73fPHi\nxZw/f54zZ87w9ddfZ7tv7dq1SUpKygh7SkoKu3btuuJsN9xwQ5Y34FDeKXPUu9Xq5nZRBy8OOzji\nPqFjffo2q8a76w7y8rca98JmjGHJkiWsXbuW6tWr07hxY/r27curr76abdtevXoRGxtLWFgY8+fP\np06dOgCEhITQvXt3goOD6dKlC82bN89235IlS/L5558zatQogoKCCA4OznhP1ctp3bo1u3fv1h+e\nerFLo/5s02fdLurgQS/bWxAiwvNf7eKDjb8x8I7qjL2vrkeeltGXWlXO4onfSyLCtG3TmLNzDg/U\neoCxTce6XNS97mV7C8IYw/Md62OM4b31BxHgWQ+Nu1IqO3eIel5o2NMYYxh/fz0A5qw/iAg810Hj\nrpSnExGmbpvK+zvfp3vt7jzT5Bm3jjpo2LNIj7sx8P6PBxGEcR3qadyV8lCeGHXQsGdjjGFcB8fK\nfe6PhwA07kp5IBFhyrYpzN0516OiDhr2HKXH3WB4/8eDgMZdKU9yadTHNhnrUX++NeyXYYxJO8f+\n7zl3x2kaz/mfr5Q3EhGmbJ3C3F2eGXXw8uvYr8YYw7P31eWhO6ozb8MhJny9W69zL4BWrVqxfPny\nLLdNnTqVwYMHExUVxZIlSzJur127Ni+99FLGx126dGHRokW5Os7EiROvus2hQ4cICAjI5eQOpUuX\nztP2uZWUlESTJk1o2LAh69atK5Rj5NaMGTO4/fbbMcZw8qTLvG+O03hD1EHDflXpcR+ocS+w6Oho\nFixYkOW2BQsWEB0dTXh4eMYTiE6dOkXp0qWzvBzAxo0bCQ8Pz9VxchN2V7Jy5Urq1KlDXFxcjk+2\nyonNZiuUWSIiIlixYgXVqlUrlP1bSUR4Y+sbHh910LDnijGGsffV5eHmjrg//9UujXs+dO3alW++\n+Ybk5GTAsWo+cuQId9xxBxERERlh37BhAx06dCApKQkR4eDBg1x77bVUqlQpy/6OHj1KixYtCA4O\nJiAggHXr1jF69GjOnz9PcHAwvXr14rnnnmPatGkZ9xk7dizTp0/Psh+bzcaIESNo1KgRgYGBzJo1\n64qPQ0QYMWIEAQEBNGjQIONZqjnNY7PZ6NevX8a2U6ZMybKv+Ph4Ro4cydKlSwkODub8+fN88skn\nNGjQgICAAEaNGpWxbenSpRk3bhxNmjTJ8pfegQMHsrxi5v79+7O92mVuNWzYEH9//3zd15WlR33e\nrnn0qN3Do6MOeo4914wxPNPecV377LQXDpuQ9qQmd/Tq5lf55c9fnLrPOuXrMKrxqMt+3tfXl8aN\nG7Ns2TIiIyNZsGAB3bt3xxhDaGgoO3fu5OLFi2zYsIGWLVuSkJDAnj17iIuLIyIiItv+Pv74Y9q1\na8fYsWOx2WycO3eO5s2bM2PGDOLj4wHHXx6dO3dm2LBh2O12FixYwObNm7O8JsycOXMoW7YsW7Zs\nITk5mYiICNq2bUv16tVzfByLFi0iPj6e7du3c/LkSRo1akSLFi1ynCc+Pp7ExER27twJwN9//51l\nX8HBwVleLvjIkSOMGjWKrVu3cuONN9K2bVuWLFlCp06d+OeffwgICMj2ZiK33XYbZcuWJT4+nuDg\nYObOnUu/fv0AGD58OKtXr872GHr06MHo0aMv+//Kk4gI/439Lx/s/oAetXvwTJNn3PbPbW5p2PPA\nGMOYex2vVzI7JgEReCHSfeNuhfTTMelhT3999GuuuYb69euzbds2Nm3axMiRI0lISGDDhg3ExcXl\neBqmUaNGDBgwgJSUFDp16kRwcHC2bfz9/fH19SUuLo7jx4/TsGFDfH19s4T9+++/Z8eOHRlv0HH6\n9Gn2799/2bCvX7+e6OhofHx8uPnmm2nZsiVbtmzJcZ4aNWqQkJDA0KFDue+++2jbtu0Vvz5btmyh\nVatWVKhQAXC8Xk5MTAydOnXCx8eHLl265Hi/gQMHMnfuXN544w0WLlzI5s2bAbL9C8HbZI56dJ1o\nxjQe4xV/XjXseZQedwMZr+fujnG/0sq6MHXq1Iknn3ySbdu2cf78+SynEMLDw4mJieHMmTPceOON\nNG3alBkzZhAXF5flHZfStWjRgpiYGL799lt69+7NiBEj6NOnT7btBg4cyLx58zh27BgDBgzI9nkR\n4c0336Rdu3a5egyXOw13uXm2b9/O8uXLmTlzJp9++mnGX2Z52TdAqVKl8PHxyfFzXbp0YcKECdx5\n552Ehobi6+sLXH3F3q5dO44fP05YWBjvvffelR622xERXo99nQ93f+hVUQccD76of4WGhoq7s9vt\nMnHpbqk26hsZu3iH2Gx2q0e6qt27d1s9goiIdOvWTYKCgmT8+PFZbl+8eLHUqFFDBgwYICIiKSkp\nUrduXSlfvrzYbLZs+zl06JCkpKSIiMiUKVNk2LBhIiJSrlw5uXjxYsZ2ycnJUqtWLalevbqkpqaK\niMjBgwelfv36IiIya9YsiYyMzLjP3r175ezZs9mOd/3114uIyBdffCFt27aV1NRUOXHihPj5+cnR\no0dznCcpKUlOnz4tIiJxcXESFBSUbb9z586VIUOGiIjIkSNHxM/PT5KSkiQ1NVXuuusuWbJkSZbj\nX85jjz0mlStXlqVLl15xu9yoVq2aJCUlXfbzrvK9dDl2u10mb54sAfMCZOKmiWK3u/6fz9wAYiUX\njdUVez4ZYxh9j+O0zKy1aSv3jgEUK+YlK4ICiI6OpnPnztmukAkPDychIYExY8YAULx4cSpWrEjV\nqlUpViz7z/nXrFnDa6+9RokSJShdujQffvghAIMGDSIwMJCQkBDmz59PyZIlad26NeXKlctxxTtw\n4EAOHTpESEgIIkKFChWyXHp5qaioKDZu3EhQUBDGGCZPnkylSpX44IMPss2TmJhI//79sdvtAEya\nNOmKX5vKlSszadIkWrdujYjQvn17IiMjr/wFTdOrVy8WLVp01dM9VzJ9+nQmT57MsWPHCAwMpH37\n9m63kpdMK/WedXoyuvFo71mpp9GX7S0gEeHVZXt5Z+0BejXx48VI1427J77Uam7Y7XZCQkL47LPP\nqFmzptXjFJrXX3+d06dP8+KLLxb6sVz1e0lEeC32NT7a/RG96vZiVKNRHhV1fdneImKMYdQ9tTEG\n3l5zAAFecuG4e5vdu3fToUMHoqKiPDrqUVFRHDhwgFWrVlk9imU8Pep5oWF3AmMMI9vVxgBvrTmA\nCLzcSePuCurVq0dCQoLVYxS6xYsXWz2CpUSEyVsm8397/o8H6z7IyEYjvTbqoGF3GmMMI9rVBhxx\nB9eMu4h49Te8KjgrTt9eiUY9O6eF3RjjA8QCiSLSwVn7dSfpcTcGZq4+AAgvd2rgMnEvVaoUp06d\nwtfX1+u/8VX+iAinTp2iVKlSVo8CaNQvx5kr9mHAHqCME/fpdowxPN22NgbDjNW/IgITo1wj7lWq\nVOHw4cMkJSVZPYpyY6VKlaJKlSpWj6FRvwKnhN0YUwW4D3gZeNIZ+3RnxhiealsLgBmrfwVcI+4l\nSpS47LMplXInmaPeu15vRoSN0Khn4qwV+1RgJHCDk/bn9tLjbgy8ucqxcp/U2fq4K+XuRIRXt7zK\n/D3zNeqXUeCwG2M6ACdEZKsxptUVthsEDALw8/Mr6GHdgjGGJ9vUwgDTVzlW7hp3pfJPo547zlix\nRwAdjTHtgVJAGWPM/4nIg5k3EpHZwGxwPEHJCcd1C8YYhrepBcYwfeV+BOGVzoEad6XySER4ZfMr\nfPzLx/Sp14enw57WqF9GgcMuImOAMQBpK/anL426tzPGMPxux5Njpq/cjwi82kXjrlRuadTzRq9j\nLyKZT8tMW7kf0LgrlRsiwqTNk/jkl0/oW68vT4U9pVG/CqeGXUTWAGucuU9PM7yN42qZaSv3Izji\n7qNxVypHGvX80RW7BbLEXWByV427UpcSESb+NJEFexfQr34/ngx9UqOeSxp2iwxv47gUcuoKx2kZ\njbtS/9KoF4yG3UJP3F0Lg2HKin0IwmtdgzTuyutljnr/+v0ZHjpco55HGnaLDbu7JsbAGz/sA4HX\numnclfcSEV7+6WUW7l2oUS8ADbsLePwux6WQb/ywD9C4K++UJeoB/RkeolHPLw27i3j8rpoY4L8a\nd+WFNOrOpWF3IUPvcpyWef37fQjwusZdeQG72Jn400QW7l3IgIABPBHyhEa9gDTsLuaxOx2nZV7/\nfh8iwn8fCNa4K4+lUS8cGnYX9NidNTHG8NryvQjwhsZdeSC72Hl508t8uu9THgp4iGEhwzTqTqJh\nd1FDWt8OwGvL9wLw325BFPcpZuVISjmNRr1wadhd2JDWt2MMTF62FxF44wGNu3J/drHz0qaX+Gzf\nZwxsMJDHGz6uUXcyDbuLG9zKsXKfvMyxcte4K3emUS8aGnY3MLjV7RgMry77BQGmaNyVG7KLnRc3\nvcjn+z7XqBcyDbub+E+r2zAGXvnuF0DjrtxL5qg/3OBhhjYcqlEvRBp2N/Joy9sAR9xFhKndgzXu\nyuXZxc4LG1/gi/1faNSLiIbdzTza8jYMMOk7x2mZaRp35cI06tbQsLuhR1o6TstMXOo4LaNxV65I\no24dDbubGtTiNgyGl5fuAYGpPYIpoXFXLiJz1AcFDuKx4Mc06kVIw+7GHm5RA4CXl+5BEKb1aKhx\nV5bTqFtPw+7mHm5RA2PgpW/3AHEad2WpzFF/JPARhgQP0ahbQMPuAQY2d6zcX/p2DyJxTI/WuKui\nZxc7EzZOYNH+RRp1i+mffg8xsHkNnr2vLt/tPMbjn8SRYrNbPZLyIpmj/mjQoxp1ixU47MaYUsaY\nzcaY7caYXcaYCc4YTOVd5rgP/VjjroqGXew8v+H5jKgPDhqsUbeYM1bsycCdIhIEBAP3GGOaOmG/\nKh8GNq/Bcx3qsWyXxl0VvvSoL/51Mf8J+o+u1F1EgcMuDmfTPiyR9ksKul+Vfw/dUZ1xaXF/7ONt\nXEzVuCvns4ud8RvGZ0R9cPBgq0dSaZxyjt0Y42OMiQdOAD+IyE85bDPIGBNrjIlNSkpyxmHVFQy4\nozrj76/H8l3HNe7K6dKjvuTXJRp1F+SUsIuITUSCgSpAY2NMQA7bzBaRMBEJq1ChgjMOq66if4Qj\n7t/v1rgr59Gouz6nXhUjIn8Da4B7nLlflX/9I6rzfFrch2jcVQHZ7DbG/TiOJb8uYXDQYI26i3LG\nVTEVjDHl0n5/LXA38EtB96ucp19EdSZ0rM8PGndVADa7jfEbxvPlgS8ZHDSY/wT/x+qR1GU4Y8Ve\nGVhtjNkBbMFxjv0bJ+xXOVHfcP+MuA+er3FXeWOz2xi3YZwj6sEadVdX4GeeisgOoKETZlGFrG+4\nP8bAuC93MXj+Nt7qFULJ4vocNXVl6VH/6sBXjqgHadRdnf6p9jJ9mvnzYmR9Vuw5zuD5W0lOtVk9\nknJhGnX3pGH3Qr0z4n6Cwf+3TeOucpQ56kOCh2jU3YiG3Uv1bubPi50CWPmLxl1ld2nUHw161OqR\nVB5o2L1Y76bVeCkt7v/RuKs0NruN5358jq8OfMVjwY9p1N2Qht3LPZgW91Uad8W/Uf864WseC36M\nR4IesXoklQ8adsWDTavxcpQj7o9+pD9Q9VaZoz604VCNuhvTsCsAejWpxsSoBqzem8SjH23lQorG\n3ZvY7Dae/fHZjKgPChxk9UiqADTsKkPPJn7/xv3/NO7eIj3q3yR8w+MNH9eoewANu8qiZxM/JnVu\nwJq9STyiK3ePZ7PbGPvj2IyoPxz4sNUjKSfQsKtsohs74r52n8bdk6VH/duEbxkWMkyj7kE07CpH\n0Y39eEXj7rFsdhvPrH8mI+oDGwy0eiTlRBp2dVk9GvvxahdH3Adp3D1Gqj2VZ9Y/w9KDSzXqHkrD\nrq6oeyNH3NftT+LhD2M17m4u1Z7K2PVjNeoeTsOurqp7Iz9e7RzI+l9PatzdWOaV+hMhT2jUPZiG\nXeXKA42q8moXjbu7So/6dwe/44mQJ3iowUNWj6QKkYZd5doDYRp3d5Q56sNDh2vUvYCGXeXJA2FV\nmZwW94EfaNxdXao9lWfW/Rv1AQEDrB5JFQENu8qzbmFVea1rED8ecMT9/EWNuyvKiPqh73gy9EmN\nuhfRsKt86Rpa5d+4f7hF4+5iLo16/4D+Vo+kipCGXeVbetw3HDjFQx9o3F1Fqj2VMevGaNS9mIZd\nFUjX0Cq83jWIjQkad1eQHvVlh5bxVOhTGnUvVeCwG2OqGmNWG2P2GGN2GWOGOWMw5T66hFbhv900\n7lZLtacyet3ojKj3C+hn9UjKIs5YsacCT4lIXaApMMQYU88J+1VupHNIFd54wBH3AfO2cO5iqtUj\neZX0qC8/tJynw57WqHu5AoddRI6KyLa0358B9gC3FnS/yv1ENXTE/aeDp3hoXqzGvYik2lMZFTMq\nI+p96/e1eiRlMaeeYzfG+AMNgZ+cuV/lPhxxD+ang7pyLwop9hRGxYzi+9++16irDE4LuzGmNPAF\n8ISI/C+Hzw8yxsQaY2KTkpKcdVjlgjo1vJU3Hghm88E/Ne6FKMWewuiY0Rp1lY1Twm6MKYEj6vNF\nZFFO24jIbBEJE5GwChUqOOOwyoV1angrU7o74t5/rsbd2TKv1EeEjdCoqyyccVWMAeYAe0TkjYKP\npDxFZLAj7lsO/Uk/jbvTpEf9h99+YETYCPrU72P1SMrFOGPFHgH0Bu40xsSn/WrvhP0qD5Ae99i0\nuP+TrHEvCI26yo3iBd2BiKwHjBNmUR4qMvhWjDE8sSCO/nO3MLd/I66/psDfel4nc9RHNhpJ73q9\nrR5JuSh95qkqEh2DbmFqj4bE/uY4564r97zRqKu80LCrItMx6Bam9WjI1t//ot/czRr3XMoc9VGN\nRmnU1VVp2FWRuj/oFqZ2D2bb73/Tb+5mzmrcryjFnsLItSMzov5gvQetHkm5AQ27KnL3B93CtB5p\ncX9f435GE7z9AAAPAUlEQVQ56VFf8fsKRjcerVFXuaZhV5boEHgL03s0JO4PjXtOUuwpjFg7IiPq\nver2snok5UY07Moy9wVW1rjnIMXmiPrK31dq1FW+aNiVpe4LrMyb0Y64931/M2cupFg9kqVSbCmM\niNGoq4LRsCvLtW/giHu8l8c9xZbC02ufZuXvKxnTeIxGXeWbhl25hPYNKjMjuiE7Dp/2yrinR33V\nH6sY03gMPev2tHok5cY07Mpl3Ju2cve2uKfYUnhq7VMadeU0GnblUu5tUJkZPR1x7/P+Zv7n4XFP\nj/rqP1bzTJNnNOrKKTTsyuXcE1CZGT1D+Dlt5e6pcb806tF1oq0eSXkIDbtySfcEVMqIe585nhf3\nFFsKT659UqOuCoWGXbmsewIqMbNXCDsTPSvu6VFf88caxjYZq1FXTqdhVy6tXf1/4957zmZOn3fv\nuF8a9R51elg9kvJAGnbl8trVr8RbvULYfcTxA1V3jXuKLYUn1zii/myTZzXqqtBo2JVbaFu/EjN7\npsV9zk9uF/eLtouOqB92RL17ne5Wj6Q8mIZduY229SvxVq9Qdh/9n1vF/aLtIk+teYo1h9fwXNPn\nNOqq0GnYlVtpU+9m3k6Le283iHvmlfpzTZ/jgdoPWD2S8gIaduV27k6L+570uJ9zzbinR33t4bUa\ndVWkNOzKLd1d72beedAR9wddMO4XbRcZvma4Rl1ZwilhN8a8b4w5YYzZ6Yz9KZUbd9V1xH3vsTMu\nFff0qMccjmFcs3EadVXknLVinwfc46R9KZVrd9W9mXd6h7D32Bl6zdlkedwvjXq3Wt0snUd5J6eE\nXURigD+dsS+l8urOOo647zt2ll5zNvH3uYuWzHHRdpEnVj+hUVeW03PsyiPcWedmZvUOdcT9vZ+K\nPO7JtmSeWP0E6xLXMb7ZeI26slSRhd0YM8gYE2uMiU1KSiqqwyov0rpORWb1DmX/8aKNe7ItmeGr\nh2dEvWutrkVyXKUup8jCLiKzRSRMRMIqVKhQVIdVXqZ1nYrM6hPK/hNFE/dLV+oadeUK9FSM8jit\na1dkdm9H3Hu++xN//VM4cU+P+vrE9Tzf7HmNunIZzrrc8RNgI1DbGHPYGPOQM/arVH61Sov7r0mO\nlbuz435p1LvU6uLU/StVEM66KiZaRCqLSAkRqSIic5yxX6UKolXtirzbJ4xfk87S04lxT7YlM2z1\nMNYnrmdC+ASNunI5eipGebSWtSrwbp8wDqTF/c8Cxj096hsSNzAhfAKda3Z20qRKOY+GXXm8LHF/\nd1O+465RV+5Cw668QstaFXivTxgHT/6Tr7gn25IZturfqEfVjCqkSZUqOA278hotalXgvb55j3tG\n1I9o1JV70LArr9K8Zta4nzqbfMXtNerKHWnYlddpXrMCc/o24uDJf+j13k+XjfuF1As8vupxjbpy\nOxp25ZXuqHlTRtx7vps97hdSLzBs9TA2HtnICxEvaNSVW9GwK691R82beL9fIw6dcsT9ZFrc01fq\n6VHvdHsniydVKm807MqrRdzuiPtvfzrOuSeePs3jqx5n09FNvBjxokZduSUNu/J6EbffxPt9G/H7\nX6fp+NnAjKhH3h5p9WhK5YuGXSkg0K8U9RouJLn4Xsqc7UVEJX1DMOW+NOzK6/194W8e/v5hDp7Z\nzcN1nuXksUCiZ28i6cyVL4VUylVp2JVXO3HuBP2X92ffX/uY0noKw5o+wPv9GvHHX+fo+e4mjv/v\ngtUjKpVnGnbltfac2kP0t9Eknk1k5t0zaVW1FQDht93E3H6NSfz7PJEzfmTXkdPWDqpUHmnYlVda\n88ca+i7ri8Hw0b0f0bRy0yyfb3abL58/Go4x0O2djazYfdyiSZXKOw278ip2sTNr+yweX/U4NcrW\n4JP7PqF2+do5blvvljJ8OSSC2yuW5uGPYpm+cj82uxTxxErlnYZdeY0/L/zJf1b8hxnxM7i3+r3M\nvWcuFa678vvvVixTioWDmhEZdAtv/LCPvu9v1h+qKpenYVdeIeZwDF2/6krssVjGNRvHK81f4dri\n1+bqvteW9GFK92Be7dKALYf+pP30dazee6KQJ1Yq/zTsyqOdTj7N2PVjGbJyCGWvKcv8++bTrVY3\njDF52o8xhu6N/PjysQjKXVuC/nO38NSn2/n7XOG8UbZSBVHc6gGUKgw2u40lvy5hetx0TiefZlDg\nIB4JfISSPiULtN86lcrwzeN3MGPVr7y95gBr9yUxol0tuoZWxadY3v6yUKqwGJGi/2FQWFiYxMbG\nFvlxlecTETYe2ciUbVP45c9faFixIWMaj6Gub12nH2vXkdOM+3IXW3/7i7qVyzDm3jo0r3lTnv81\noFRuGWO2ikjYVbfTsCtPYBc7MYdjmL1jNj+f/JlK11fiqdCnaOffrlBDKyJ8+/NRXvnuFw7/dZ6g\nKmUZ3Pp22tS9mWK6gldOVqRhN8bcA0wDfID3ROSVK22vYVfOcvyf4yz5dQmLf11M4tlEbi19Kw81\neIjI2yILfNolL5JTbXyxNZF31h7g9z/P4Vf+OrqFVqFLaBVuKZe7H9IqdTVFFnZjjA+wD2gDHAa2\nANEisvty99Gwq/xKsaew/6/9bDiygdV/rObnpJ8RhCaVmtClVhfurnY3JYqVsGy+VJud73Ye45PN\nv7PhwCmMgeCq5birTkVa1a5InUo3UNxHr1lQ+VOUYW8GPC8i7dI+HgMgIpMud5/8hv23//3GiXNZ\nLzPLaX4hl7fl8r453+TkY+Qg14+tAF+DonhsBTluij2FY/8c48jZIyScTmD3qd0k2xzXkQf4BtCy\nakvaV2+PXxm/7Du02B9/nmNxXCIr9xxn+2HHyxKUKlGMepXLULtSGW4tV4rKZa+lzLUluD71L8qc\nPYBBT994uiq1Qynre3O+7pvbsDvjqphbgT8yfXwYaOKE/Wbz0e6PWLh3YWHsWrk431K++JXx44Ha\nDxB4UyAhN4dQ8bqKVo91RVXLX8fjd9Xk8btqcuLMBTYeOMWOw6fZcfhvvt91jFP/OC6VrGN+Z37J\nl/E1ZyyeWBWFHclzCGzdtVCP4Yyw57TEyLb0MsYMAgYB+Pnlb3X1YN0HaeffLpdDZR8rpx+iFWS7\nHI+b22MU5Lg53uSZj83H+FDxuoq5fjKRq6p4Qykig28lMvjWjNsupNg4eWAblZY8hr3Y9expMgVb\n8VIWTqmKgl/tRoV+DGeE/TBQNdPHVYAjl24kIrOB2eA4FZOfA/mX9ce/rH9+7qqUyyl1ag9VvnwA\nSpSCft9Q1/c2q0dSHsIZP8XZAtQ0xlQ3xpQEegBfOWG/SnmuYzvhg/uhuCPqaNSVExV4xS4iqcaY\nx4DlOC53fF9EdhV4MqU81bGf4YOOGnVVaJzykgIishRY6ox9KeXR0qNe4lro+7VGXRUKvaBWqaKi\nUVdFRF8ETKmicOxnxzn1Etc5Tr+Ur2H1RMqD6YpdqcKmUVdFTMOuVGE6uiMt6tdr1FWR0bArVViO\n7oAPO6ZF/WuNuioyGnalCsPR7Zmirit1VbQ07Eo529Ht8GFkpqhXt3oi5WU07Eo509HtjksaS5bW\nqCvLaNiVcpb0qF9zg+M6dY26sohex66UM2SOer9v4EZ/qydSXkxX7EoV1JF4jbpyKRp2pQriSLzj\nB6XXlNGoK5ehp2KUyq9sUa9m9URKAbpiVyp/NOrKhWnYlcqrI3GOJx9p1JWL0lMxSuXFkTjHSr1U\nWeirUVeuSVfsSuWWRl25CV2xK5Ubidvgo06OqPf7Fsr5WT2RUpelK3alrkajrtyMhl2pK8mIejmN\nunIbGnalLidxG3yYHvVvNOrKbRQo7MaYbsaYXcYYuzEmzFlDKWW5xK2OqF+rK3Xlfgq6Yt8JdAZi\nnDCLUq4hcSt8GJUp6lWtnkipPCnQVTEisgfAGOOcaZSyWnrUr7vRcUmjRl25Ife63HHta7Dzc6un\nUJ7s79+hdEWNunJrVw27MWYFUCmHT40VkS9zeyBjzCBgEICfXz7PV5auCBVq5+++SuXGrWHQegyU\nrWL1JErlmxGRgu/EmDXA0yISm5vtw8LCJDY2V5sqpZRKY4zZKiJXvVBFL3dUSikPU9DLHaOMMYeB\nZsC3xpjlzhlLKaVUfhX0qpjFwGInzaKUUsoJ9FSMUkp5GA27Ukp5GA27Ukp5GA27Ukp5GA27Ukp5\nGKc8QSnPBzUmCfityA9ccDcBJ60eoojpY/Z83vZ4wX0fczURqXC1jSwJu7syxsTm5llfnkQfs+fz\ntscLnv+Y9VSMUkp5GA27Ukp5GA173sy2egAL6GP2fN72eMHDH7OeY1dKKQ+jK3allPIwGvZ8MMY8\nbYwRY8xNVs9S2IwxrxljfjHG7DDGLDbGlLN6psJijLnHGLPXGPOrMWa01fMUNmNMVWPMamPMnrQ3\npR9m9UxFxRjjY4yJM8Z8Y/UshUHDnkfGmKpAG+B3q2cpIj8AASISCOwDxlg8T6EwxvgAM4F7gXpA\ntDGmnrVTFbpU4CkRqQs0BYZ4wWNONwzYY/UQhUXDnndTgJGAV/xwQkS+F5HUtA83AZ76nnGNgV9F\nJEFELgILgEiLZypUInJURLal/f4MjtDdau1Uhc8YUwW4D3jP6lkKi4Y9D4wxHYFEEdlu9SwWGQB8\nZ/UQheRW4I9MHx/GCyKXzhjjDzQEfrJ2kiIxFcfizG71IIWlQG+04Ymu9ObdwDNA26KdqPDl5g3L\njTFjcfzTfX5RzlaETA63ecW/yowxpYEvgCdE5H9Wz1OYjDEdgBMistUY08rqeQqLhv0SInJ3Trcb\nYxoA1YHtxhhwnJLYZoxpLCLHinBEp7vcY05njOkLdADuEs+9PvYwUDXTx1WAIxbNUmSMMSVwRH2+\niCyyep4iEAF0NMa0B0oBZYwx/yciD1o8l1Ppdez5ZIw5BISJiDu+kFCuGWPuAd4AWopIktXzFBZj\nTHEcPxy+C0gEtgA9RWSXpYMVIuNYoXwA/CkiT1g9T1FLW7E/LSIdrJ7F2fQcu7qaGcANwA/GmHhj\nzDtWD1QY0n5A/BiwHMcPET/15KiniQB6A3em/b+NT1vJKjenK3allPIwumJXSikPo2FXSikPo2FX\nSikPo2FXSikPo2FXSikPo2FXSikPo2FXSikPo2FXSikP8/9zhQ8doz8tmwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1110b0f28>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = gluon.loss.Langford()\n",
    "\n",
    "with ag.record():    # start recording\n",
    "    theloss = loss(output, nd.ones_like(output))\n",
    "theloss.backward()   # and compute the gradient\n",
    "\n",
    "plt.plot(output.asnumpy(), theloss.asnumpy(), label='VW style loss')\n",
    "plt.plot(output.asnumpy(), output.grad.asnumpy(), label='Gradient')\n",
    "# now compute the loss for y=-1\n",
    "theloss = loss(output, -thelabel)\n",
    "plt.plot(output.asnumpy(), theloss.asnumpy(), label='VW style loss for y=-1')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Multiclass Classification \n",
    "## Multiclass Softmax\n",
    "\n",
    "One way of dealing with multiple classes is to turn it into $n$ binary classification problems. That is, we simply test: 'is it class 1', 'is it class 2', ... 'is it class n'. In theory this sounds like a splendid idea. After all, this should be just as easy as determining which class it is. Unfortunately, that's not quite the case. Imagine the situation where none of the $n$ classifiers wants to take responsibility. Or imagine the case where more than one claims that it's his turn. Obviously there has to be a better way. Indeed, there is.\n",
    "\n",
    "If we have a vector $f \\in \\mathbb{R}^n$ of scores, where the coordinate, say $f_i$ is large whenever we think that the correct class is $i$, then we can map $f$ into a probability vector via\n",
    "\n",
    "$$p(y=i|f) \\propto \\exp(f_i) \\text{ and hence } p(y=i|f) = \\frac{\\exp(f_i)}{\\sum_j \\exp(f_j)}$$\n",
    "\n",
    "Here the normalization by $\\sum_j \\exp(f_j)$ is needed such that all the terms sum up to 1. Consequently the negative log-likelihood $-\\log p(y|f)$, i.e. the quantity that we would want to minimize in this case is given by \n",
    "\n",
    "$$-\\log p(y=i|f) = \\log \\left[\\sum_{j} \\exp(f_j)\\right] - f_i$$\n",
    "\n",
    "In ``gluon`` the relevant function is [mxnet.gluon.loss.SoftmaxCrossEntropyLoss](http://mxnet.io/api/python/gluon.html#mxnet.gluon.loss.SoftmaxCrossEntropyLoss). Let's check that this is correct."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Softmax loss is 1.7977911233901978.\n",
      "Class 4 has negative log-likelihood 1.7977910041809082.\n"
     ]
    }
   ],
   "source": [
    "loss = gluon.loss.SoftmaxCrossEntropyLoss()\n",
    "\n",
    "f = nd.random_normal(shape=(1,10))\n",
    "y = nd.array([4]) #class 4 is true\n",
    "\n",
    "print('Softmax loss is {}.'.format(loss(f,y).asscalar()))\n",
    "\n",
    "# now compute this by hand\n",
    "\n",
    "p = nd.exp(f)\n",
    "p = p / nd.sum(p)\n",
    "print('Class 4 has negative log-likelihood {}.'.format(-nd.log(p[0,4]).asscalar()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The softmax loss has a rather nice property that is worth pointing out: its gradient is given by the difference between the conditional class probabilities $p(y=i|f)$ and the indicator vector $e_j$. This can be derived via \n",
    "\n",
    "$$\\partial_{f_i} \\log \\sum_j \\exp(f_j) = \\frac{\\exp(f_i)}{\\sum_j \\exp(f_j)} = p(y=i|f)$$\n",
    "\n",
    "Such a result seems to be too good to be true by chance. In fact, it holds for *every* member of a larger family of distributions, called the [Exponential Family](https://en.wikipedia.org/wiki/Exponential_family). More specifically, the derivative of the associated normalization is the expected value of the associated embedding. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## MaxMargin Loss\n",
    "\n",
    "The soft-margin loss function allowed us to distinguish between two classes with a margin of separation. That is, as long as $y f(x) \\geq 1$ we incurred no loss, whereas for smaller values of the margin (and for misclassifications) a loss is incurred. The obvious question is how to generalize this to more than two classes. One possibility is to treat things as many binary classification problems, but this is a bad idea, since tie-breaking can be tricky. An alternative is to require that the correct class be recognized with a safe margin relative to all the other classes as follows: $f(x,y) \\geq f(x,y') + 1$ for all $y' \\neq y$. Clearly this would do the trick, and we can design a loss function via\n",
    "\n",
    "$$l(y,f) = \\mathrm{max}\\left[0, \\mathrm{max}_{y' \\neq y} \\left[f(x,y') - f(x,y) + 1\\right]\\right]$$\n",
    "\n",
    "This looks awkward since we have two nested maxima (the outer one is needed to ensure that we don't get negative values for our loss function). A cleaner (albeit slightly wasteful) way of writing this out is to define some function $\\Delta(y,y')$ where $\\Delta(y,y') = 1$ if $y \\neq y'$ and $\\Delta(y,y) = 0$. This is a 0-1 loss. In this case the above equation can be rewritten as:\n",
    "\n",
    "$$l(y,f) = \\mathrm{max}_{y'} \\left[f(x,y') - f(x,y) + \\Delta(y,y')\\right]$$\n",
    "\n",
    "Note that the function $l$ is convex in $f$ (once upon a time when people were using kernels this was a big deal since it meant that the entire optimization problem was convex ...). More importantly for us here is the fact that we now have a parameter, the loss $\\Delta$ and an obvious question is what would happen if we changed it a bit. \n",
    "Let's take some intuition from the real world. Assume that you're driving on a road with a steep cliff on one side and an incline on the other. \n",
    "\n",
    "![](img/road-cliff.jpg)\n",
    "\n",
    "Any sensible driver will try to stay as far away from the cliff while hugging the shoulder that corresponds to the incline. This is the case since mistakes on the incline are much more benign (scratched rims) than those on the steep cliff (likely death). In other words, a good driver will pick a margin between alternatives that is commensurate with the cost of making a mistake. [Taskar, Guestrin and Koller](http://dl.acm.org/citation.cfm?id=2981349) (TKG) in 2003 realized the same thing and decided to make $\\Delta$ cost sensitive (they did lots of other things related to dynamic programming). The result is that the very same loss function as above now allows for misclassification-dependent confidence margins. Obviously this is something that we would also want in our machine learning arsenal. Enter `MaxMargin`. By default it uses the 0-1 loss above (and it automagically infers the size), but if you provide it with a suitable matrix `delta`, it will use the latter. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Function values for 3 problems \n",
      "[[ 0.84843999  0.85705417 -0.28716376 -0.65270543]\n",
      " [-0.56867689 -0.35533145  0.86864537  0.07883889]\n",
      " [ 0.50960332  0.80499649 -0.44336858  2.44341731]]\n",
      "<NDArray 3x4 @cpu(0)>\n",
      "Loss function values \n",
      "[ 0.99138576  1.78980649  3.88678598]\n",
      "<NDArray 3 @cpu(0)>\n",
      "Instantiated loss matrix \n",
      "[[ 0.  1.  1.  1.]\n",
      " [ 1.  0.  1.  1.]\n",
      " [ 1.  1.  0.  1.]\n",
      " [ 1.  1.  1.  0.]]\n",
      "<NDArray 4x4 @cpu(0)>\n",
      "Instantiated loss matrix \n",
      "[[ 0.  1.  1.  4.]\n",
      " [ 1.  0.  1.  4.]\n",
      " [ 1.  1.  0.  4.]\n",
      " [ 1.  1.  1.  0.]]\n",
      "<NDArray 4x4 @cpu(0)>\n",
      "Function values for 3 problems \n",
      "[[ 0.84843999  0.85705417 -0.28716376 -0.65270543]\n",
      " [-0.56867689 -0.35533145  0.86864537  0.07883889]\n",
      " [ 0.50960332  0.80499649 -0.44336858  2.44341731]]\n",
      "<NDArray 3x4 @cpu(0)>\n",
      "Loss function values \n",
      "[ 2.49024034  1.78980649  6.88678598]\n",
      "<NDArray 3 @cpu(0)>\n"
     ]
    }
   ],
   "source": [
    "# plain vanilla loss \n",
    "loss = gluon.loss.MaxMargin()\n",
    "\n",
    "# some classes (4 class problem)\n",
    "label = nd.array([1,3,2])\n",
    "output = nd.random_normal(shape=(3,4))\n",
    "\n",
    "print('Function values for 3 problems {}'.format(output))\n",
    "theloss = loss(output, label)\n",
    "print('Loss function values {}'.format(theloss))\n",
    "print('Instantiated loss matrix {}'.format(loss._delta))\n",
    "\n",
    "# now make things more interesting by changing the loss matrix \n",
    "delta = nd.array(loss._delta) #call copy constructor\n",
    "delta[0,3] = 4\n",
    "delta[1,3] = 4\n",
    "delta[2,3] = 4\n",
    "loss = gluon.loss.MaxMargin(delta)\n",
    "print('Instantiated loss matrix {}'.format(loss._delta))\n",
    "print('Function values for 3 problems {}'.format(output))\n",
    "theloss = loss(output, label)\n",
    "print('Loss function values {}'.format(theloss))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Information Theory Primer\n",
    "\n",
    "## Entropy\n",
    "\n",
    "Sometimes we care about probabilities rather than just labels. In particular, we might want to measure the distance between distributions. For that we need some basics about probabilities, such as the [entropy](https://en.wikipedia.org/wiki/Entropy_(information_theory)). In a nutshell, the entropy of a random variable is the amount of surprise we encounter each time we sample from it. For instance, the entropy of the constant function is zero, since we already know what's coming. The entropy of a fair coin being tossed is 1 bit. We have no idea what's going to happen (it's a fair coin after all) and there are only two possible outcomes. If we had a biased coin, e.g. one that produces heads with probability 0.9 and tails with probability 0.1, the surprise would be less (after all, most of the time we see a head). Correspondingly its entropy should be lower. On the other hand, a dice with 6 possible outcomes should have a higher degree of surprise. Without further ado, let's define the entropy function:\n",
    "\n",
    "$$H[p] := \\sum_x -p(x) \\log p(x)$$\n",
    "\n",
    "This works well for discrete outcomes. For densities we use\n",
    "\n",
    "$$H[p] := \\int -p(x) \\log p(x) dx$$\n",
    "\n",
    "We can check that for a fair coin the entropy is given by $H[p] = -2 \\cdot 0.5 \\log 0.5 = \\log 2$. Information theorists often measure the information in 'nats' rather than bits. The difference is the base of the logarithm. It's easy to convert: 1 nat is $\\log_2 e \\approx 1.44$ bit. More generally, for a uniform distribution over $N$ outcomes it is $H[p] = \\log N$. One of the fundamental theorems in information theory is that for a distribution $p$, we need at least $H[p]$ nats to encode it. \n",
    "\n",
    "There are a number of useful properties for the entropy that are employed in machine learning: \n",
    "\n",
    "* Often when estimating distributions we want to find the one with the largest entropy that fits the requirements. That's in line with our desire to restrict our estimates as little as possible beyond what we actually observe.\n",
    "* The entropy is a concave function. That is, for two distributions $p$ and $q$, the mixture of both has higher entropy: $H[\\lambda p + (1-\\lambda) q] \\geq \\lambda H[p] + (1-\\lambda) H[q]$. To prove this, simply note that the function $-x \\log x$ is concave. \n",
    "* When we have independent random variables, say $x$ and $y$, then the entropy of the joint distribution is the sum of the individual entropies. This follows simply from the fact that $\\log p(x) q(y) = \\log p(x) + \\log q(y)$. \n",
    "* For dependent random variables the joint entropy is lower than that of the individual terms. This can be seen as follows:\n",
    "\n",
    "$$\\begin{eqnarray}\n",
    "H[p(x,y)] = & \\int -p(x,y) \\log p(x,y) \\\\\n",
    "= & \\int -p(x,y) [\\log p(x) p(y)] dx dy + \\int p(x,y) \\log \\frac{p(x) p(y)}{p(x,y)} dx dy \\\\\n",
    "\\leq & H[p(x)] + H[p(y)] + \\log \\int p(x,y) \\frac{p(x) p(y)}{p(x,y)} dx dy \\\\\n",
    "= & H[p(x)] + H[p(y)]\n",
    "\\end{eqnarray}$$\n",
    "\n",
    "Here the inequality follows from the fact that $\\log x$ is a concave function, hence the expectation of the logarithm is less than the logarithm of the expectation. Intuitively this result is straightforward - if $x$ and $y$ are dependent on each other, then knowing $x$ should tell us some more about $y$. Therefore, the joint entropy of $x$ and $y$ should be lower than the sum of the individual entropies. \n",
    "\n",
    "This leads us to the notion of mutual information. It is given by the difference between joint and and independent entropies, i.e. $I(x,y) := H[p(x)] + H[p(y)] - H[p(x,y)]$. Basically it's the amount of information that we save. For instance, a light switch and a (functioning) light bulb are strongly correlated - knowing one tells us all about the other. The entropy of the joint is 1 bit (if it's on with probability 0.5), but the sum of the entropies of switch and bulb individually is 2 bit. Hence the mutual information is 1 bit. \n",
    "\n",
    "## Kullback Leibler Divergence\n",
    "\n",
    "This brings us to the KL divergence. It measures how close two distributions are. One way of defining such a quantity is to ask how many extra bits one would have to spend to encode data drawn from $p$ when using a code tuned for $q$. If we assume for a fact that it takes $-\\log p(x)$ nat to optimally encode $x$, then the penalty from using the 'wrong' code is given by \n",
    "\n",
    "$$D(p\\|q) = \\sum_x p(x) [\\log p(x) - \\log q(x)]$$\n",
    "\n",
    "For densities the quantity is defined analogously, i.e. $\\int p(x) [\\log p(x) - \\log q(x)] dx$. The first thing to prove is that this is actually a distance. For that we need to show that $D(p\\|q) \\geq 0$ with equality only for $p=q$. To see the latter, simply plug $p=q$ into the definition. To see the former, we rewrite $D$ in the same way as above, using convexity, this time of $-\\log x$. \n",
    "\n",
    "$$D(p\\|q) = \\sum_x -p(x) \\log \\frac{q(x)}{p(x)} \\geq -\\log \\sum_x p(x) \\frac{q(x)}{p(x)} = 0$$\n",
    "\n",
    "As an aside, to see that $H[p]$ can be achieved, indeed, quantize all $p(x)$ into bins of the next largest fraction of $2$, e.g. $0.2$ goes into the bin of $\\frac{1}{4}$. It is clear that the sum over those bins is no smaller than $1$ and no larger than $2$. Moreover, we can arrange them into a tree, where at level $l$ the bins are of size $2^{1-l}$. Then we simply index these bins according to their position of the tree. Each $x$ will require $\\lceil \\log_2 p(x) \\rceil$ bits (whatever is left over, we simply discard). In sum this is no more than $\\log_2 H[p] + 1$. To tighten the bound, simply send $N$ symbols. Since they can be encoded using at most $N \\log_2 H[p] + 1$ bit, the code becomes increasingly efficient with only $1/N$ waste. This proves that such a code can be found. That it's impossible to do any better is a consequence of $D(p\\|q) \\geq 0$.  \n",
    "Note that our construction relied on very long codes for efficiency. This is a real problem in practice. [Turbo codes (https://en.wikipedia.org/wiki/Turbo_code) are one of the techniques to address this, e.g. for mobile communications. \n",
    "\n",
    "After this long detour, let's finally get to the KL divergence as a loss function. It generalizes the multiclass softmax as follows: instead of having just a single possible true class, it uses a probability distribution as reference. That is\n",
    "\n",
    "$$\\log(f,y) = \\sum_i y_i (log(y_i) - f_i)$$\n",
    "\n",
    "Here $f_i$ is assume to be a probability distribution (or we can set a flag to transform the output into one beforehand). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "[ 0.05661784]\n",
      "<NDArray 1 @cpu(0)>\n",
      "\n",
      "[  1.39972247e-08]\n",
      "<NDArray 1 @cpu(0)>\n",
      "\n",
      "[ 0.22255933]\n",
      "<NDArray 1 @cpu(0)>\n",
      "\n",
      "[ 2.22559333]\n",
      "<NDArray 1 @cpu(0)>\n"
     ]
    }
   ],
   "source": [
    "loss = gluon.loss.KLDivLoss()\n",
    "\n",
    "# generate some random probability distribution\n",
    "f = nd.random_normal(shape=(1,10))\n",
    "p = nd.exp(f)\n",
    "p = p / nd.sum(p)\n",
    "\n",
    "# generate some target distribution\n",
    "y = nd.random_normal(shape=(1,10))\n",
    "y = nd.exp(y)\n",
    "y = y / nd.sum(y)\n",
    "\n",
    "z = nd.zeros_like(y)\n",
    "z[0,3] = 1\n",
    "\n",
    "# distance between our estimate p and the 'true' distribution y\n",
    "print(loss(nd.log(p), y))\n",
    "# distance to itself - should be zero\n",
    "print(loss(nd.log(p), p))\n",
    "# equivalent of logistic loss with class 3 up to normalization over domain, i.e. 1/10\n",
    "# note that this is VERY DIFFERENT from information theory but a traditional choice\n",
    "# in deep learning\n",
    "print(loss(nd.log(p), z))\n",
    "print(-nd.log(p[0,3]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## KL Divergence Estimator\n",
    "\n",
    "Loss functions can also be used for other purposes, such as estimating interesting properties about a distribution. For instance, we might want to *estimate* the KL divergence between two distributions directly. Unfortunately this is difficult, since it requires density estimation and even the ratio between two densities $p(x)/q(x)$ to begin with. A rather neat trick was suggested by [Nguyen, Wainwright and Jordan](http://dept.stat.lsa.umich.edu/~xuanlong/Papers/Nguyen-Wainwright-Jordan-aos09.pdf) (NWJ) in 2009 when they realized that convex duality can be used to estimate such quantities rather directly. Before we dive in, we briefly need to explain what the Fenchel-Legendre dual of a function is:\n",
    "\n",
    "$$F^*(z) = \\mathrm{sup}_x x^\\top z - F(x)$$\n",
    "\n",
    "$f^*$ basically compares a line with slope $z$ to the function $F$ and measures the smallest distance to that line. It has the neat property that its dual is the function itself, i.e. $F^{**} = F$, provided that the function is convex and well-behaved. NWJ used this to derive estimators of the [F-divergence](https://en.wikipedia.org/wiki/F-divergence) between distributions. The latter is defined as \n",
    "\n",
    "$$D_F(p\\|q) := \\int dq(x) F\\left(\\frac{p(x)}{q(x)}\\right)$$\n",
    "\n",
    "Plugging in duality, we can rewrite this as an optimization problem in terms of $F^*$ (remember, the dual of the dual is the original function, at least for well-behaved convex ones). That is, we obtain\n",
    "\n",
    "$$\\begin{eqnarray}\n",
    "D_F(p\\|q) = & \\int dq(x) \\sup_G \\left[\\frac{p(x)}{q(x)} G(x) - F^*(G(x))\\right] \\\\\n",
    "=& \\sup_G \\left[\\int dp(x) G(x) - \\int dq(x) F^*(G(x))\\right]\n",
    "\\end{eqnarray}$$\n",
    "\n",
    "Skipping over details of when and whether this is possible in general, we now have the difference in expectations over two different function - $G(x)$ and $F^*(G(x))$ for two different distributions, namely $p$ and $q$. These can be replaced by empirical estimates (aka sample averages) and it now looks very much like a classification problem, albeit with a weird kind of loss function. In particular, the KL-divergence has $F(x) = x \\log x$. After quite some algebra, and a substitution due to the fact that $G$ needs to be nonnegative, which lets us set $G(x) = \\exp(g(x))$ we arrive at the following problem:\n",
    "\n",
    "$$D_F(p\\|q) = \\sup_g \\left[\\int dp(x) \\exp(g(x)) - \\int dq(x) (g(x) + 1)\\right]$$\n",
    "\n",
    "This looks just like a classification problem with a weird loss function and with $p$ and $q$ substituted for classes $-1$ and $1$. The 'loss function' `DualKL` in `Gluon` provides this functionality. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt4VPW97/H3N/cb94SLJBKoyEVAxBQvtNSKeK+2e9dj\ntSrVnu2x6tGe1u5ePMe63ad97NPu2tqzrbXi5bQer61Va7vd3nqhohUQEAQBEUi4JRASEnLPfM8f\nM8EAQ0gyM1kzk8/reeaZWWt+s9Z3In7Wmt/6rbXM3RERkfSVEXQBIiKSWAp6EZE0p6AXEUlzCnoR\nkTSnoBcRSXMKehGRNKegFxFJcwp6EZE0p6AXEUlzWUEXAFBcXOzl5eVBlyEiklKWL1++x91LjtUu\nKYK+vLycZcuWBV2GiEhKMbOtvWmnrhsRkTSnoBcRSXMKehGRNJcUffTRtLe3U1VVRUtLS9ClpIW8\nvDxKS0vJzs4OuhQRGWBJG/RVVVUMGTKE8vJyzCzoclKau7N3716qqqqYOHFi0OWIyABL2q6blpYW\nRo0apZCPAzNj1KhR+nUkMkglbdADCvk40t9SZPBK6qAXEUlnP31lI0s27kn4ehT0PSgqKkr4Op5+\n+mmmTZvGpz/96YSvS0SSR0t7Jz95dQNvb6lN+LoU9AFbvHgx9913H6+//nqv2nd0dCS4IhEZCB/U\nNOIOJ44ZkvB1Kej7aOvWrSxYsIBZs2axYMECtm3bBoT3zGfMmMHJJ5/M/PnzAVi7di1z585l9uzZ\nzJo1i40bNx6yrLvuuoslS5Zwww038I1vfIOWlhauvfZaZs6cySmnnHIw/B955BEuu+wyPvOZz3Du\nuecesowtW7YwdepUFi1axKxZs/j85z9PU1PTAPwlRCQWG3c3AjB5TOJ7DpJ2eGV3//LCWt7bsT+u\ny5x+3FC++5mT+vy5m2++mWuuuYZFixbx0EMPccstt/C73/2Ou+66i5deeonx48dTV1cHwP3338+t\nt97KF7/4Rdra2ujs7DxkWXfccQevvfYaP/rRj6ioqODf/u3fAHj33XdZv3495557Lhs2bABg6dKl\nrF69mpEjRx5R0/vvv8/ixYuZN28e1113Hffddx+33XZbn7+biAycjdUNZGUY5aMKE74u7dH30dKl\nS7nyyisBuPrqq1myZAkA8+bN40tf+hK//OUvDwb6GWecwfe//31+8IMfsHXrVvLz83tc9pIlS7j6\n6qsBmDp1KhMmTDgY9AsXLowa8gBlZWXMmzcPgKuuuupgTSKSvDbubqS8uJCcrMTHcErs0fdnz3ug\ndA1bvP/++3nrrbd48cUXmT17NitXruTKK6/ktNNO48UXX+S8887jwQcf5Oyzzz7qstz9qO8VFh59\nq3/40EkNpRRJfpuqG5kyNvH989CLPXoze8jMqs1sTZT3bjMzN7PiyLSZ2b1mtsnMVpvZnEQUHaQz\nzzyTJ554AoDHHnuMT3ziEwB88MEHnHbaadx1110UFxdTWVnJ5s2bmTRpErfccguXXHIJq1ev7nHZ\n8+fP57HHHgNgw4YNbNu2jSlTphyzpm3btrF06VIAHn/88YM1iUhyamnvZMveA0wegAOx0Luum0eA\n8w+faWZlwEJgW7fZFwCTI4/rgZ/HXmJwmpqaKC0tPfj48Y9/zL333svDDz/MrFmz+NWvfsVPf/pT\nAL7xjW8wc+ZMZsyYwfz58zn55JN58sknmTFjBrNnz2b9+vVcc801Pa7vxhtvpLOzk5kzZ3L55Zfz\nyCOPkJube8w6p02bxqOPPsqsWbOora3lK1/5Sly+v4gkxod7DhBymDw68QdiAayn7oKDjczKgd+7\n+4xu854B/hV4Dqhw9z1m9gvgT+7+eKTN+8BZ7r6zp+VXVFT44TceWbduHdOmTevbtxmEtmzZwsUX\nX8yaNUf84DqC/qYiyeH5VTu45fF3+I+vfpKpY4f2ezlmttzdK47Vrl9HAczsEmC7u6867K3xQGW3\n6arIvGjLuN7MlpnZspqamv6UISKSkjbubiAzw5hYnPgRN9CPoDezAuB24I5ob0eZF/Ung7s/4O4V\n7l5RUnLMWx7KUZSXl/dqb15EksfG3Y1MGFVAblbmgKyvP6NuPgZMBFZFRneUAivMbC7hPfiybm1L\ngR2xFikikk427G4YkBOluvR5j97d33X30e5e7u7lhMN9jrvvAp4HromMvjkdqD9W/7yIyGDS3BYe\ncRNL33xf9WZ45ePAUmCKmVWZ2Zd7aP4HYDOwCfglcGNcqhQRSRMbqxsIOUwbNzBDK6EXXTfufsUx\n3i/v9tqBm2IvS0QkPa3f1QDAlGTaox/MdJliEYm39TsbyM/O5PiRBQO2TgV9wHSZYpHBZf2u/Zw4\npojMjIG7VImCvo+S7TLFAN/73veYMmUK55xzDldccQU/+tGPEvxXEJH+cHfW72oY0AOxkCIXNeOP\n34Jd78Z3mWNnwgV39/ljyXaZ4uXLl/PEE0/wzjvv0NHRwZw5czj11FP78xcRkQSraWyl9kDbgF3M\nrIv26Pso2S5T/Ne//pXPfe5zFBQUMHToUC655JK4fVcRia/3Iwdipw7giBtIlT36fux5D5RkvEyx\niCSn9TsjQT/AXTfao++jZLtM8fz583n22Wdpbm6moaGBF154IQ7fUkQSYd2u/YweksvIwpwBXW9q\n7NEHpOsyxV2+9rWvce+993Ldddfxwx/+kJKSEh5++GEgfJnijRs34u4sWLCAk08+mbvvvptf//rX\nZGdnM3bsWO64I9rlgT5y4403csMNNzBz5kyysrJ6dZniOXPmcPnllzN79mwmTJjAJz/5ydi/uIgk\nxPu7Gpg6bmD35qGXlylONF2mOH7uvPNOioqKot4zVn9TkeB0dIaYfsdLfGleOd+5MD7/Hyb0MsUi\nItI3H+45QFtniKkDPOIG1HWTdu68886gSxCRKD669MHAB31S79EnQ7dSutDfUiRY63ftJzPDOGGA\nbh/YXdIGfV5eHnv37lVAxYG7s3fvXvLy8oIuRWTQWrtjP5NHFw3YzUa6S9qum9LSUqqqqtBtBuMj\nLy/vkBFEIjJw3J012+s5a8roQNaftEGfnZ3NxIkTgy5DRCRm1Q2t7Gls46TjBn5oJSRx142ISLpY\ns70egBnjhwWy/t7cYeohM6s2szXd5v3QzNab2Woze9bMhnd779tmtsnM3jez8xJVuIhIqlizfT9m\nMC2Ak6Wgd3v0jwDnHzbvZWCGu88CNgDfBjCz6cAXgJMin7nPzAb+yIOISBJZs6OeicWFFOUG01t+\nzKB3978AtYfN+09377oDxptA11G+S4En3L3V3T8kfO/YuXGsV0Qk5by3Yz8zjgum2wbi00d/HfDH\nyOvxQGW396oi80REBqXaA21sr2tmxvhgum0gxqA3s9uBDuCxrllRmkUdCG9m15vZMjNbpiGUIpKu\n1u6IHIhNxT16M1sEXAx80T86q6kKKOvWrBTYEe3z7v6Au1e4e0VJSUl/yxARSWprtu8H4KRUC3oz\nOx/4JnCJuzd1e+t54AtmlmtmE4HJwN9jL1NEJDWt2VFP2ch8hhVkB1bDMQ8Bm9njwFlAsZlVAd8l\nPMomF3g5cnejN939Bndfa2ZPAe8R7tK5yd07oy9ZRCT9rd1eH2i3DfQi6N39iiizF/fQ/nvA92Ip\nSkQkHexvaWfL3iYuqyg7duME0pmxIiIJ8t6OcP/89IAufdBFQS8ikiCrKusAmBXQpQ+6KOhFRBJk\nVVUdZSPzGVXU872fE01BLyKSICu31TG7bETQZSjoRUQSoXp/CzvqW5hdNvzYjRNMQS8ikgArI/3z\ns8uC7Z8HBb2ISEKsrKwjK8MCPSO2i4JeRCQBVlXVMXXcEPKyg79Su4JeRCTOQiFndWV9UvTPg4Je\nRCTuNu9ppKG1g5NLFfQiImnpnW3hA7GnHK+gFxFJSysr6xiSm8Wk4qKgSwEU9CIicbeqqo5ZZcPI\nyIh2L6aBp6AXEYmj5rZO1u9sSJr+eVDQi4jE1aqqOjpCTkV58Jc+6KKgFxGJo2VbagGYc7yCXkQk\nLb29ZR8njilieEFO0KUcdMygN7OHzKzazNZ0mzfSzF42s42R5xGR+WZm95rZJjNbbWZzElm8iEgy\n6Qw5K7buo6J8ZNClHKI3e/SPAOcfNu9bwKvuPhl4NTINcAHhG4JPBq4Hfh6fMkVEkt+G3Q00tHbw\n8STqn4deBL27/wWoPWz2pcCjkdePAp/tNv//etibwHAzGxevYkVEkllX/3zFhNTbo49mjLvvBIg8\nj47MHw9UdmtXFZknIpL23t6yjzFDcykdkR90KYeI98HYaGcHeNSGZteb2TIzW1ZTUxPnMkREBt7y\nSP+8WXKcKNWlv0G/u6tLJvJcHZlfBZR1a1cK7Ii2AHd/wN0r3L2ipKSkn2WIiCSH7XXNbK9r5uMT\nkqt/Hvof9M8DiyKvFwHPdZt/TWT0zelAfVcXj4hIOjvYP59kI24Aso7VwMweB84Cis2sCvgucDfw\nlJl9GdgGXBZp/gfgQmAT0ARcm4CaRUSSzrIt+yjMyWTq2CFBl3KEYwa9u19xlLcWRGnrwE2xFiUi\nkmre3lLLnAkjyMpMvvNQk68iEZEUs7exlfW7Gjh90qigS4lKQS8iEqM3N4f758/4mIJeRCQtLd28\nh8KcTGaOHxZ0KVEp6EVEYvTGB3uZO3Ek2UnYPw8KehGRmOze38LmmgNJ220DCnoRkZi8uXkvAGdM\nKg64kqNT0IuIxOCNTXsZmpfF9OOGBl3KUSnoRURisHTzXk6bNIrMJLkReDQKehGRfqra18S22ibO\nSNLx810U9CIi/bT0g3D//JknKOhFRNLSkk17GFWYw4mjk+/6Nt0p6EVE+iEUcv66cQ/zTywhI4n7\n50FBLyLSL2t21FN7oI35JybvsMouCnoRkX74y4bwnfE+OTn5b5ykoBcR6Yc/b6hhxvihFBflBl3K\nMSnoRUT6aH9LOyu21fGpE5N/bx4U9CIiffbGpj10hpz5KdBtAzEGvZn9DzNba2ZrzOxxM8szs4lm\n9paZbTSzJ80sJ17Fiogkgz9v2ENRbhZzkvBG4NH0O+jNbDxwC1Dh7jOATOALwA+Ae9x9MrAP+HI8\nChURSQbuzl821HDmx0Yl7WWJDxdrlVlAvpllAQXATuBs4JnI+48Cn41xHSIiSeODmka21zXzqSmp\n0W0DMQS9u28HfgRsIxzw9cByoM7dOyLNqoDxsRYpIpIsXl1XDcBZU0YHXEnvxdJ1MwK4FJgIHAcU\nAhdEaepH+fz1ZrbMzJbV1NT0twwRkQH1yrrdTB83lPHD84Mupddi6bo5B/jQ3WvcvR34LXAmMDzS\nlQNQCuyI9mF3f8DdK9y9oqQkdX4CicjgtbexleVb97Fw+pigS+mTWIJ+G3C6mRWYmQELgPeA14HP\nR9osAp6LrUQRkeTw2vpqQs7gCXp3f4vwQdcVwLuRZT0AfBP4mpltAkYBi+NQp4hI4F5Zt5uxQ/M4\nKYnvJhVN1rGbHJ27fxf47mGzNwNzY1muiEiyaWnv5C8b9vCPp44n3ImROlJjEKiISMCWfrCX5vZO\nFk4fG3QpfaagFxHphZfX7aYwJ5PTJ40MupQ+U9CLiBxDKOS88t5uPjWlhNyszKDL6TMFvYjIMazY\nto/qhlbOOyn1um1AQS8icky/X72TnKwMFkxLrWGVXRT0IiI9CIWcP67ZyVknllCUG9NAxcAo6EVE\nerB82z5272/lolnjgi6l3xT0IiI9eDHFu21AQS8iclShkPOHd3fy6Smp220DCnoRkaNatjU82uai\nWccFXUpMFPQiIkfx4uod5GZlsGBq6lx7PhoFvYhIFO2dIX6/eidnTx1NYQp324CCXkQkqr9urGHv\ngTb+YU5p0KXETEEvIhLFb1ZsZ0RBNp86MfVvjKSgFxE5TH1zOy+/t5tLTj6OnKzUj8nU/wYiInH2\nx3d30tYRSotuG1DQi4gc4bcrtjOppJBZpcOCLiUuYgp6MxtuZs+Y2XozW2dmZ5jZSDN72cw2Rp5H\nxKtYEZFEq6xt4u9bavnHOaUpdyepo4l1j/6nwH+4+1TgZGAd8C3gVXefDLwamRYRSQm/WVEFwKWz\nU/skqe76HfRmNhSYT+Tm3+7e5u51wKXAo5FmjwKfjbVIEZGB0Blynny7kk9OLqZ0REHQ5cRNLHv0\nk4Aa4GEze8fMHjSzQmCMu+8EiDyn9illIjJo/On9anbWt3Dl3OODLiWuYgn6LGAO8HN3PwU4QB+6\naczsejNbZmbLampqYihDRCQ+Hv/7NoqLcjlneupeqTKaWIK+Cqhy97ci088QDv7dZjYOIPJcHe3D\n7v6Au1e4e0VJSeqfkCAiqW1nfTOvra/mv1SUkp2ZXgMS+/1t3H0XUGlmUyKzFgDvAc8DiyLzFgHP\nxVShiMgAePLtSkIOV6RZtw2Eu19i8d+Bx8wsB9gMXEt44/GUmX0Z2AZcFuM6REQSqvtB2LKR6XMQ\ntktMQe/uK4GKKG8tiGW5IiID6dV1u9lZ38IdF08PupSESK+OKBGRfnjobx8yfng+C9PsIGwXBb2I\nDGprd9Tz5uZaFp05gaw0OwjbJT2/lYhILz38ty0U5GRyeUX6HYTtoqAXkUGruqGF51fu4POnljKs\nIDvochJGQS8ig9Zjb26jrTPEl84sD7qUhFLQi8ig1NLeyWNvbWXB1NFMKikKupyEUtCLyKD05NuV\n7Gls45/mTwq6lIRT0IvIoNPWEeIXf/6AigkjOG3iyKDLSTgFvYgMOr97Zzs76lu46ewT0ubmIj1R\n0IvIoNIZcn7+5w+YMX4oZ504OC6oqKAXkUHlD+/u5MM9B7jprMGxNw8KehEZRDpDzs9e28jHSgo5\n76SxQZczYBT0IjJoPLdyOxt2N/K1hVPIyBgce/OgoBeRQaKtI8Q9r2zgpOOGcsGMwbM3Dwp6ERkk\nnlxWSWVtM7edN7j25kFBLyKDQHNbJz97dSNzy0cOmpE23SnoRSTtPfS3D6luaOW286YMmpE23cUc\n9GaWaWbvmNnvI9MTzewtM9toZk9GbjMoIhKI6v0t/Pvrm1g4fQxzB8FZsNHEY4/+VmBdt+kfAPe4\n+2RgH/DlOKxDRKRffvjS+7R3hrj9wmlBlxKYmILezEqBi4AHI9MGnA08E2nyKPDZWNYhItJf71bV\n88yKKq6bN5Hy4sKgywlMrHv0PwH+GQhFpkcBde7eEZmuAsZH+6CZXW9my8xsWU1NTYxliIgcyt35\nlxfWMrIgh5vOPiHocgLV76A3s4uBandf3n12lKYe7fPu/oC7V7h7RUnJ4DsKLiKJ9ew721m2dR+3\nnTeFoXnpe/eo3siK4bPzgEvM7EIgDxhKeA9/uJllRfbqS4EdsZcpItJ7tQfa+N8vrmPO8cO5vKIs\n6HIC1+89enf/truXuns58AXgNXf/IvA68PlIs0XAczFXKSLSB9//wzr2N7fz/X+YOehOjoomEePo\nvwl8zcw2Ee6zX5yAdYiIRPXGB3t4ZnkV/zR/ElPHDg26nKQQS9fNQe7+J+BPkdebgbnxWK6ISF80\nt3Vy+7NrOH5kAbcumBx0OUkjLkEvIpIMfvAf6/lwzwH+3389jbzszKDLSRq6BIKIpIW/bdrDI29s\n4UtnlnPmCcVBl5NUFPQikvLqm9u57elVTCop5JvnTw26nKSjrhsRSWnuzh3PraG6oZXffOVM8nPU\nZXM47dGLSEp78u1Knlu5g1vOnszssuFBl5OUFPQikrLW7dzPd59fyydOKObmQX6Zg54o6EUkJTW2\ndnDTYysYmp/NPZfPJlMnRh2Vgl5EUk4o5Hz9qZVs2XuAe79wCiVDcoMuKakp6EUk5dzzygZeWrub\n2y+azhkfGxV0OUlPQS8iKeWFVTv42WubuLyijOvmlQddTkpQ0ItIylhVWcdtT6/i4+Uj+NfPzhiU\n93/tDwW9iKSEzTWNXPvI25QMyeXnV51KTpbiq7f0lxKRpLervoWrF/8dA3715dMoLtLB175Q0ItI\nUqtrauOah96irqmNR66dy8RBfO/X/tIlEEQkadU3tXP14r+zZU8Tj1z7cWaWDgu6pJSkoBeRpFTX\n1MZVi99iw65Gfn7VHF2RMgYKehFJOvsOtPHFB99iU00jv7j6VD49dXTQJaW0fvfRm1mZmb1uZuvM\nbK2Z3RqZP9LMXjazjZHnEfErV0TS3fa6Zi77xVI21TTygEI+LmI5GNsBfN3dpwGnAzeZ2XTgW8Cr\n7j4ZeDUyLSJyTOt37ecf7vsbu+tbePTauZw1RSEfD/0Oenff6e4rIq8bgHXAeOBS4NFIs0eBz8Za\npIikv6Uf7OWy+5cC8NQNZ+jSBnEUl+GVZlYOnAK8BYxx950Q3hgAUTfJZna9mS0zs2U1NTXxKENE\nUpC78+gbW7h68VuMGZrHb2+cx7RxQ4MuK63EfDDWzIqA3wBfdff9vT0l2d0fAB4AqKio8FjrEJHU\n09rRyf/63RqeWlbFOdNGc8/lsxmSlx10WWknpqA3s2zCIf+Yu/82Mnu3mY1z951mNg6ojrVIEUk/\nlbVN3Pz4O6yqrOOWs0/gq+ecSIauKZ8Q/Q56C++6LwbWufuPu731PLAIuDvy/FxMFYpI2nlu5Xb+\n57NrwOD+q+Zw/oxxQZeU1mLZo58HXA28a2YrI/O+QzjgnzKzLwPbgMtiK1FE0kVDSzt3Pv8ev1lR\nxakTRvCTy2dTNrIg6LLSXr+D3t2XAEf7nbWgv8sVkfT0ynu7+Z+/W0N1Qwu3LJjMLWefQFamLrc1\nEHRmrIgk1J7GVu58fi2/X72TqWOHcP/VpzK7bHjQZQ0qCnoRSYj2zhC/fnMrP3llI81tnXx94Yn8\nt099TNeRD4CCXkTi7vX11fzri++xueYAn5xczHc/M50TRg8JuqxBS0EvInHz9pZa7nl5A298sJdJ\nxYUsXlTB2VNH65Z/AVPQi0jMlm+t5Z6XN7Jk0x6Ki3K54+LpXHX6BHXTJAkFvYj0SyjkvLq+msVL\nNvPm5lpGFeZw+4XTuOr0CeTnZAZdnnSjoBeRPjnQ2sEzy6t4+G8fsmVvE8cNy+M7F07lqtMnUJCj\nSElG+q8iIsfk7rxTWcfTyyp5YdVOGls7mF02nJ+dO4XzZ4wlW+Phk5qCXkSOqmpfEy+u3snTy6vY\nVN1IfnYmF80axxVzj+fUCbqnUKpQ0IvIISprm/jjmp28+O4uVlXWAXDqhBH84B9nctGs4yjKVWyk\nGv0XExnk2jtDrNi6jz9vqOFP79fw3s79AMwcP4xvnj+VC2aMpby4MOAqJRYKepFBJhRyNlY38vct\ntbyxaQ9LNu6hobWDzAzj1Akj+NYFU7lwxjiOH6WLjaULBb1Immtp7+S9nftZtqWWv3+4j2Vba6lr\nagdg3LA8Lpo1jrOmlHDmCcUM1U0/0pKCXiSNNLeFQ33N9nre3V7Pmu31bKxupDMUvonbxOJCzp0+\nho+Xj+S0iaMoG5mvs1YHgdQO+i1L4E93Q3YBZOdDTmH4dU4BZBdGngsi8/OjzCv4aDozB/QPXlLE\n/pZ2NlU3HvGo3NeER27MOaowhxnjh3HOtDHMGD+MOROGM3pIXrCFSyBSO+hDneFH4y5oa4L2Jmg7\nEH7ubOvbsizzKBuKY2wgjphXeNh7kecMnSkovdfWEWJ7XTNV+5qorG2mcl8TVfuaqaxtompfE3sa\nP/r3nZOVwaTiQmaWDuNzp4znpOOGMrN0GGOH5mlvXYBUD/pJnwo/ounsCAd+9/Bva4L2A5Hn5m6v\nDxy2oWj+6HXbAWisOfJzHupbrVl5h24UevoF0n0DcbBNtM9FnrNy9WskBYRCzv6Wduqa2tl7oJXq\n/a3s3t9CdUPrR4/IdO2BQ3dUsjKM44bnUzYyn3OmjWHCqEImjy7ihNFFlI0sIFP3WpUeJCzozex8\n4KdAJvCgu9+dqHVFlZkFmUMhb2j8l+0OHa2HbUQO20AcsYFpirKxaYIDNVAX2YB0te9o6Vs9lhHe\nUGTnH6Xb6vBfIN3bHOuXS6F+jUS0d4Y40NpBY2sHB1o7I88d3eZ1sL+lg7qmduqa2qhrbmdfU9vB\n6frmdiJd5YfIyjBKhuQyekgupSMKmDNhBKOH5DJ+eD5lIwsoG1nA2KF5CnPpt4QEvZllAv8OLASq\ngLfN7Hl3fy8R6xtwZpCdF34UjIz/8kOdh24U2puP8suj6dANyeG/XNqboan2yM95Z9/qyczpoYuq\nhw1Edn4Pn+t6Pz/qr5FQyOkIOR2hEO2dTkdniM6Q0x4Kv27vDL/X0RlpF5kXbhOZ3xk62L6lPURr\nR+cRz60dIVraw8+t7YdOt7R30tzeeTDU2zp69yuuKDeL4QXZDC/IZkRBDqUjChien82IgmyGFeQw\noiCbkYU5jB6Sx+ihuYwsyCFDIS4JlKg9+rnAJnffDGBmTwCXAnEN+jXb63ni7W0ABw9Add9h8oMT\nfth09/YeZV7P7Yjaznv47KFtDml3yGJ7qtOAIqCo9981E8gHz++2fneyaCc71Equt5ATaiHXW8gN\ntZDjLZF5zeR6KzmhZnJCH7XJaWsht7WZHG8l1xvIC9WQ663keQu5tJDnLeTRt2MjITdayKGJPJrJ\npYlcmjyXZs+hidzwPM/r9jqXZnJoJi/yuttnDmkTnt9xlH/iOZkZ5GZlkJudGXnOIC8r8+BzYWEW\neVmZ5GVnUJibRVFuFoWRR1FuZrfXHz26pnVpXkk2iQr68UBlt+kq4LTuDczseuB6gOOPP75fK9lZ\n38If393VbZkHXx0xz45oAxaZe+i8g/Udsb5D2h1c7tHX1X05UUqLuq7e1nn48qN9tudl5GLkAsMO\naWgGZEKGQYYZGWZY5HVmxkevu963bq8zMiATJ5e2cOh7eGOS6y0HNwY5oVZyvZm8yEYmJ7KByQmF\nNyzZoRaGh5oZ7S3kdB4gO7SX7FAzWZ0tZHU2k+kdR/4ReuAZ2YS6uqyy87Gcwsijr79Autpnf9RF\nlpUPGQp1SX6JCvpov0MP6Z109weABwAqKiqi9Fwe28LpY1g4fWF/PiqpqrM9ehdV1APuTVh7E5nR\nur1a6qGzNtzSAAAES0lEQVRh15HHUejjP8XsnrqvjnZcpDdDfgshM1sH2SUuEhX0VUBZt+lSYEeC\n1iWDSWY25A8PP+LN/aMD6tEOnEc9uH6UEV0JHfJ7tJFZ3Y59aMivdJOooH8bmGxmE4HtwBeAKxO0\nLpH4MAuHYE6CrvEStyG/jdBYfWSbvv4a6cuQ36hdWz2cU6Ihv0klIUHv7h1mdjPwEuFDgg+5+9pE\nrEskZQQy5PewobtRu70SOOS31ycdashvIiVsHL27/wH4Q6KWLyLdJNOQ36jnlHRrG5chv7lxOumw\n90N+U1lqnxkrIgMjIxNyh4Qf8eYePn5xrA1Ej8dOIp9rqoX2qkPbdDT3sSCLvqHoaQPRl3NKMgf+\nCqEKehEJllm4Tz8rNzHLD4XCYd/fkw67z9u//ciNTqi9b/VkZB+6gTj1Wjjz5sR89wgFvYikt4yM\n8J50TiFQEv/l93HI7xHzikbHv6bDKOhFRGKRyCG/caLT+kRE0pyCXkQkzSnoRUTSnIJeRCTNKehF\nRNKcgl5EJM0p6EVE0pyCXkQkzVn329sFVoRZDbA16Dr6oRjYE3QRA0zfeXDQd04NE9z9mKf7JkXQ\npyozW+buFUHXMZD0nQcHfef0oq4bEZE0p6AXEUlzCvrYPBB0AQHQdx4c9J3TiProRUTSnPboRUTS\nnII+TszsNjNzMysOupZEM7Mfmtl6M1ttZs+aWfJeiDsGZna+mb1vZpvM7FtB15NoZlZmZq+b2Toz\nW2tmtwZd00Axs0wze8fMfh90LYmgoI8DMysDFgLbgq5lgLwMzHD3WcAG4NsB1xN3ZpYJ/DtwATAd\nuMLMpgdbVcJ1AF9392nA6cBNg+A7d7kVWBd0EYmioI+Pe4B/BgbFAQ93/09374hMvgmUBllPgswF\nNrn7ZndvA54ALg24poRy953uviLyuoFw8I0PtqrEM7NS4CLgwaBrSRQFfYzM7BJgu7uvCrqWgFwH\n/DHoIhJgPFDZbbqKQRB6XcysHDgFeCvYSgbETwjvqIWCLiRRdM/YXjCzV4CxUd66HfgOcO7AVpR4\nPX1nd38u0uZ2wj/3HxvI2gaIRZk3KH6xmVkR8Bvgq+6+P+h6EsnMLgaq3X25mZ0VdD2JoqDvBXc/\nJ9p8M5sJTARWmRmEuzBWmNlcd981gCXG3dG+cxczWwRcDCzw9ByjWwWUdZsuBXYEVMuAMbNswiH/\nmLv/Nuh6BsA84BIzuxDIA4aa2a/d/aqA64orjaOPIzPbAlS4e6pdGKlPzOx84MfAp9y9Juh6EsHM\nsggfaF4AbAfeBq5097WBFpZAFt5beRSodfevBl3PQIvs0d/m7hcHXUu8qY9e+uP/AEOAl81spZnd\nH3RB8RY52Hwz8BLhg5JPpXPIR8wDrgbOjvx3XRnZ05UUpz16EZE0pz16EZE0p6AXEUlzCnoRkTSn\noBcRSXMKehGRNKegFxFJcwp6EZE0p6AXEUlz/x9ljr9iITsvYwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x111001da0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# we broke the output data previously\n",
    "output = nd.arange(-5,5,0.01)\n",
    "output.attach_grad() # we need the gradient\n",
    "\n",
    "loss = gluon.loss.DualKL()\n",
    "\n",
    "lossp = loss(output, -nd.ones_like(output))\n",
    "lossq = loss(output, nd.ones_like(output))\n",
    "\n",
    "plt.plot(output.asnumpy(), lossp.asnumpy(), label='Loss for p')\n",
    "plt.plot(output.asnumpy(), lossq.asnumpy(), label='Loss for q')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Relative Novelty Detection\n",
    "\n",
    "In some cases estimating a density or even a ratio of densities is not really what we want. Instead, we would like to find the *most typical* or the *most unusual* observation in a dataset. Unfortunately, these things are not really well defined. Before going into measure theory, we need some culinary support. \n",
    "\n",
    "|![](img/doughnut.jpg)|![](img/berliner.jpg)|\n",
    "|:---------------:|:---------------:|\n",
    "|Doughnut|Jelly Doughnut|\n",
    "\n",
    "Now imagine that we have two different pastry-shaped distributions - a doughnut shaped one and one that looks like a jelly doughnut (also called 'Berliner' in Europe). These two couldn't be more different from each other. Any data occurring in the donut hole (or far away in its periphery) is novel, whereas for the jelly doughnut only the data far away is novel. Yet we can transform one into the other, simply by messing with the radius in polar coordinates, e.g. via a new radial coordinate $r' = 1/r$. Hence, what once was novel is now no longer novel since we stretched out the center of the poor jelly doughnut so much that its density becomes infinitesimally low. In mathematics terms, this means that novelty is sensitive to the *measure* of the domain where it's defined. This is bad, since we usually don't know this measure. For a 3D space, there are still assumptions of what is considered reasonable (stretching out a poor jelly doughnut probably is not). But for arbitrary domains (database records, books, images, movies, TCP/IP logs) it's pretty hard to define what is reasonable. However, we all know that something that looks just like what we've seen before is probably reasonable ... but that's just like defining what is novely by saying that it's novel if it looks novel. Ouch!\n",
    "\n",
    "Here's a mathematically more sound way: we use data to define an implicit reference measure. E.g. for server logs we could use past data as a reference measure, such that we can ask the question whether something looks out of order relative to what we've seen in the past. Or for images, whether there's one that stands out relative to past images. Or even for pixels within an image. Mathematically this means that we care mostly about $p(x)/q(x)$ whenever $p(x)/q(x)$ is particularly small. For large ratios things are just fine. This is precisely what [Smola, Le and Teo](http://proceedings.mlr.press/v5/smola09a/smola09a.pdf) (SLT) in 2009 did in their Relative Novelty Detection paper. They used the same reasoning as NGW but with a different F-divergence function: \n",
    "\n",
    "$$F\\left(\\frac{p(x)}{q(x)}\\right) = \\mathrm{max}\\left(0, \\rho - \\log \\frac{p(x)}{q(x)}\\right)$$\n",
    "\n",
    "Here $\\rho$ serves as a threshold to decide whether the density ratio is too low. Anything lower than $\\exp(\\rho)$ is too small. This actually allows us to focus on both the very typical and the very atypical aspects of the data, simply by picking a very small and a very large $\\rho$ respectively. Note that for very large $\\rho$ this is just the *reverse KL Divergence*, i.e. pretty much the same thing as what NGW were using. Again, skipping over the tedious mathematical details of computing the dual of $F$ and of substituting (we have the same problem of nonnegativity) we arrive at the following loss function:\n",
    "\n",
    "$$l(y,f) = \\begin{cases}\n",
    "   \\exp(f - \\rho) & \\text{ if } y = -1 \\\\\n",
    "   -f-1 & \\text{ if } y = 1 \\text{ and } f > 0 \\\\\n",
    "   \\exp(f) & \\text{ if } y = 1 \\text{ and } f <= 0\n",
    "\\end{cases}$$\n",
    "\n",
    "'Training' with this loss function will give us precisely the relative novelty detector that we want. All we need to do now is threshold it at $\\rho$ to get the desired output. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD9CAYAAACoXlzKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8VFX+//HXmXRCIIAgSBKaSDGEFkCkKKCoK9jXgqKI\nLiL2XXXddVddf7pfd/XrKuraRVb4ooLYxYJiwUUk9A6REkINoRNSJnN+f9yAAQIJmXInk/fz8ZjH\n5N65c+5nhgfv3Jx77znGWouIiEQOj9sFiIhIYCnYRUQijIJdRCTCKNhFRCKMgl1EJMIo2EVEIkxA\ngt0Yc48xZqkxZokxZpIxJj4Q7YqIyInzO9iNMc2BO4FMa206EAVc7W+7IiJSPYHqiokGEowx0UAd\nYFOA2hURkRPkd7BbazcCTwE5wGZgt7X2S3/bFRGR6on2twFjTAPgYqAVsAuYbIy5zlo74YjtRgGj\nABITE7u3b9/e312LiNQqc+fO3W6tbVzZdn4HO3AOsNZamwdgjJkKnAkcFuzW2leAVwAyMzNtVlZW\nAHYtIlJ7GGPWV2W7QPSx5wBnGGPqGGMMMAhYHoB2RUSkGgLRxz4bmALMAxaXtfmKv+2KiEj1BKIr\nBmvtw8DDgWhLRET8E5BgD4SSkhJyc3MpLCx0u5QaLz4+npSUFGJiYtwuRURcEDbBnpubS1JSEi1b\ntsTpqpfqsNaSn59Pbm4urVq1crscEXFB2IwVU1hYSKNGjRTqfjLG0KhRI/3lI1KLhU2wAwr1ANH3\nKFK7hVWwu61u3bpB38fkyZPp0KEDAwYMCPq+RCR87Cks4fFPl7F2+/6g70vBHmKvv/46//73v5kx\nY0aVtvd6vUGuSERC4cfV23n1h7Xk7S0K+r4U7JVYv349gwYNIiMjg0GDBpGTkwM4R97p6el07tyZ\n/v37A7B06VJ69uxJly5dyMjIYPXq1Ye19eijjzJz5kxGjx7NfffdR2FhITfeeCOdOnWia9euh8L+\nzTff5Le//S1Dhw5l8ODBh7Wxbt062rdvzw033EBGRgZXXHEFBQUFIfgmRMQfM1ZuIyk+mm5pycHf\nmbU25I/u3bvbIy1btuyodaGWmJh41LohQ4bYN99801pr7euvv24vvvhia6216enpNjc311pr7c6d\nO6211t5+++12woQJ1lpri4qKbEFBwVHtnXXWWXbOnDnWWmufeuopO2LECGuttcuXL7epqan2wIED\ndty4cbZ58+Y2Pz//qPevXbvWAnbmzJnWWmtvvPFG++STTx61XTh8nyLi8Pl8tufjX9lbJ2T51Q6Q\nZauQsWFzuWN5f/t4Kcs27Qlomx1PqcfDQ08/4ffNmjWLqVOnAjB8+HDuv/9+APr06cOIESO48sor\nueyyywDo3bs3jz/+OLm5uVx22WW0bdv2uG3PnDmTO+64A4D27dvTokULVq1aBcC5555Lw4YNK3xf\namoqffr0AeC6665j7Nix3HvvvSf82UQkNJZv3svWPUWc3a5JSPanrpgTdPCKk5deeonHHnuMDRs2\n0KVLF/Lz8xk2bBgfffQRCQkJnHfeeXzzzTfHbcv5BVyxxMTESms41rKIhJcZK7cBcPZplQ7MGBBh\necRenSPrYDnzzDN5++23GT58OBMnTqRv374A/PLLL/Tq1YtevXrx8ccfs2HDBnbv3k3r1q258847\nWbNmDYsWLWLgwIHHbLt///5MnDiRgQMHsmrVKnJycmjXrh3z5s07bk05OTnMmjWL3r17M2nSpEM1\niUh4mr58Kxkp9WlSLzSzhuqIvZyCggJSUlIOPZ5++mnGjh3LuHHjyMjI4K233uLZZ58F4L777qNT\np06kp6fTv39/OnfuzDvvvEN6ejpdunRhxYoVXH/99cfd35gxYygtLaVTp05cddVVvPnmm8TFxVVa\nZ4cOHRg/fjwZGRns2LGDW2+9NSCfX0QCb9veQhZs2MW5HU4O2T7N8boDgqWi8diXL19Ohw4dQl5L\nTbNu3TqGDBnCkiVLjrudvk+R8PDOnBz++N5ipt3Vjw7N6vnVljFmrrU2s7LtdMQuIhJEXy3bRvPk\nBNo3TQrZPhXsNUzLli0rPVoXkfBwoLiUmdl5nNvx5JBe5KBgFxEJkh+zt1NY4uOcEPavg4JdRCRo\nvlq2laS4aHq2qvielGBRsIuIBIHPZ/l6xVbOateY2OjQRq2CXUQkCOZv2Mn2fcWc2zG03TCgYD+M\nhu0VkUD5dNEWYqM9DGwfmmEEylOwh5iG7RWJfD6f5bPFm+nftjFJ8aGfezggwW6MSTbGTDHGrDDG\nLDfG9A5Eu+Eg3IbtBXj88cdp164d55xzDtdccw1PPfVUkL8FETkR8zfsZMueQi7MaOpOAVUZArKy\nBzAeuLns51gg+Xjba9je6g/bm5WVZdPT0+3+/fvt7t27bZs2bTRsr0iY+dtHS23bBz+zew4UB7Rd\nQjVsrzGmHtAfGFH2i6IYKPar0WkPwJbF/pZ2uKad4IInTvht4TZs7w8//MCll15KnTp1ALjoootO\n+DOJSPC43Q0DgemKaQ3kAeOMMfONMa8ZY4495mwNF47D9opI+DjYDTMko5lrNQRi2N5ooBtwh7V2\ntjHmWeAB4K/lNzLGjAJGAaSlpR2/xWocWQdLuA3b279/f0aMGMEDDzyA1+vl448/5pZbbgn45xaR\n6vlk0WZioz0M6hD6q2EOCkSw5wK51trZZctTcIL9MNbaV4BXwBndMQD7DbiDw/Ye9Pvf/56xY8cy\ncuRInnzySRo3bsy4ceMAZ9je1atXY61l0KBBdO7cmSeeeIIJEyYQExND06ZNeeihh467vzFjxjB6\n9Gg6depEdHR0lYbt7datG1dddRVdunShRYsW9OvXz/8PLiIBUeqzTFu8xdVuGAjQsL3GmB9wTp6u\nNMY8AiRaa+871vYatjdwHnnkEerWrXvU1Hj6PkVC78fs7Vz72myeu6YrQzufEvD2qzpsb6BmULoD\nmGiMiQXWADcGqF0RkRpj6ryNJMVFu3K3aXkBCXZr7QKg0t8iEniPPPKI2yWICFBQ7GXaks0MzTiF\n+JgoV2vRnaciIgHwxdItFBSXclm35m6XEl7BHoj+ftH3KOKGqfM2ktIggR4tQztEb0XCJtjj4+PJ\nz89XKPnJWkt+fj7x8aGZDV1EYOueQn7M3s6lXZvj8bh/n0mgTp76LSUlhdzcXPLy8twupcaLj48/\n7LJNEQmuDxdsxGfh0q7ud8NAGAV7TEwMrVq1crsMEZETYq3l3axcuqYl07px8If+roqw6YoREamJ\n5qzbSfa2fVzTs5I76kNIwS4i4odJP+eQFBft6tgwR1Kwi4hU066CYj5dvJlLuzWnTmzY9Gwr2EVE\nquu9eRsp9vq4ukf4dMOAgl1EpFqstUz6OYcuqcl0PKWe2+UcRsEuIlINB0+aDusVXkfroGAXEamW\n8f9dR7348DppepCCXUTkBG3cdYDPl27hml5pYXXS9CAFu4jICfrPrHUAXN+7pZtlHJOCXUTkBBQU\ne5k0O4fzT29K8+QEt8upkIJdROQEvDdvI3sKvYzs29LtUo5JwS4iUkU+n2Xcj2vpnFKfbmkN3C7n\nmBTsIiJV9PWKbazJ28/Ivq0wxv3heY9FwS4iUgXWWp6fkU1qwwQu7BR+lziWp2AXEamC//6Sz8IN\nuxh9Vhuio8I7OsO7OhGRMPH8N9k0SYrjiu7hP4lNwILdGBNljJlvjPkkUG2KiISDuet3MmtNPqP6\ntyYuOsrtcioVyCP2u4DlAWxPRCQs/HtGNg3qxITVZBrHE5BgN8akABcCrwWiPRGRcDE/Zydfr9jG\nTX1bkRgXfsMHVCRQR+zPAPcDvgC1JyISFp76ciWNEmO5sU/NmZPZ72A3xgwBtllr51ay3ShjTJYx\nJisvL8/f3YqIBN2P2dv5MTuf2wacWmOO1iEwR+x9gIuMMeuAt4GBxpgJR25krX3FWptprc1s3Lhx\nAHYrIhI81lqe/GIlzerHh+WY68fjd7Bba/9krU2x1rYErga+sdZe53dlIiIumr58Gws27OKuQW2J\njwn/K2HK03XsIiJHKCn18cS05bQ6KZHLa8B160cKaKeRtfZb4NtAtikiEmoTflrPL3n7ee36TGLC\n/C7TitS8ikVEgmjn/mKemb6avqeexKAOTdwup1oU7CIi5TwzfRV7C0v465COYT2C4/Eo2EVEyqze\nupcJs3O4tlcL2jVNcrucalOwi4jgTKLx4PtLSIqP5p5zT3O7HL8o2EVEgMlzN/Dzuh38+YIONEyM\ndbscvyjYRaTW276viL9/toKerRry28yad3njkRTsIlLrPfbJMgqKvfz90k419oRpeQp2EanVvl6+\nlQ8WbOLWs0/l1CZ13S4nIBTsIlJr7dhfzB/fW0z7pkncNqCN2+UETM0ZrkxEJICstfzlg8XsPlDM\nWzf1rBEzI1WVjthFpFb6aOEmPlu8hXvOPY0Ozeq5XU5AKdhFpNbJyS/gLx8soVtaMrf0j5wumIMU\n7CJSqxR5S7l90jwM8OzVXYny1PyrYI6kPnYRqVX+57MVLMrdzcvDu5PasI7b5QSFjthFpNb4bPFm\n3vzvOkb2acV5pzd1u5ygUbCLSK2wfPMe7p28kC6pyTxwQXu3ywkqBbuIRLzt+4q4eXwW9eJjeGV4\nd2KjIzv61McuIhGt2OtjzIR5bN9XxOTRvWlSL97tkoJOwS4iEctay5/fX8zP63bw3DVdyUhJdruk\nkIjsv0dEpFb7x+crmTI3l7vPacvQzqe4XU7I+B3sxphUY8wMY8xyY8xSY8xdgShMRMQfr36/hpe+\n+4XhZ7TgrkFt3S4npALRFeMF/mCtnWeMSQLmGmO+stYuC0DbIiInbMrcXB7/bDkXdmrGIxedHhFD\n8Z4Iv4/YrbWbrbXzyn7eCywHmvvbrohIdbw3N5f7piyk76kn8fRVnSPyztLKBLSP3RjTEugKzA5k\nuyIiVTFlbi73TlnImW0a8er1mRE1YuOJCFiwG2PqAu8Bd1tr91Tw+ihjTJYxJisvLy9QuxURAeCd\nOTmHjtRfv6EHCbG1M9QhQMFujInBCfWJ1tqpFW1jrX3FWptprc1s3LhxIHYrIoK1lhdmZPPH9xbT\nr21jXr0+k/iY2hvqEICTp8Y5K/E6sNxa+7T/JYmIVE2pz/Lox0sZP2s9l3Q5hX9e0Tni7yqtikBc\nFdMHGA4sNsYsKFv3Z2vtZwFoW0SkQvuLvPz+3QV8sXQro/q35oHz2+OphSdKK+J3sFtrZwL6NkUk\nZHLyCxj1Vhartu7loSEdGdm3ldslhRUNKSAiNcrM1du57f/mATB+ZE/6tdU5uyMp2EWkRvCW+nh+\nRjZjv17NqU3q8ur1mbRolOh2WWFJwS4iYW/jrgPc/fZ85qzbyWVdm/PoJenUjVN8HYu+GREJW9Za\nPlq4ib9+sASfhWeu6sIlXXVje2UU7CISljbtOsCD7y9mxso8uqYl88xVXdT1UkUKdhEJK6U+y8TZ\n6/nHtBX4LPx1SEdGnNmyVo75Ul0KdhEJG7N+yefRT5axfPMe+rU9ib9f2onUhnXcLqvGUbCLiOty\n8gv4+2fL+XzpFponJ/DcNV0ZktGs1g23GygKdhFxTe7OAp7/Jpspc3OJifLwh3NP43f9W9f6sV78\npWAXkZDbuOsAL8zIZnLWBgyGa3ulMWbAqZxcCyaaDgUFu4iEzPycnbw+cy3TlmzBY+DqHmmMGdCG\nZvUT3C4toijYRSSoirylfLVsK2/MXMu8nF0kxUdzU99WjDizJackK9CDQcEuIkGxYsse3p2Ty/vz\nc9lZUEKLRnV4ZGhHrshM1V2jQaZvV0QCZuOuA0xbvJmPFm5iUe5uYqIMgzs25coeqfQ99SRdix4i\nCnYR8cuGHQV8vmQLny7ezIINuwA4/ZR6PDy0Ixd3aU7DxFiXK6x9FOwickIKS0r5ee0Ovl2Zx3er\ntvFL3n4A0pvX4/7z2/Gb9Ga0PEm3/rtJwS4ix1XkLWVR7m5+XruDn9fuYPbafApLfMRGezijdSOG\n9WrBOR2aaByXMKJgF5FDrLVs2VPIko17WLBhJ3PW7mRB7i6KvT4A2japy9U90jirXWPOaNWIhFjd\nSBSOFOwitZS31MeGnQdYsXkPSzbtZsnGPSzZuJv8/cUARHsMpzevzw29W9CjZUMyWzZUf3kNoWAX\niXB7CkvIyS/gl7x9ZG/bd+h53fYCikudI/Foj6HtyUkMbN+ETin1Of2U+nRsVk9H5DWUgl2kBvP5\nLDsKitm2p4hNuw6Qu7OA3J0H2HDweUcBewq9h7b3GEhrWIdTm9RlQPsmnNq4LqednES7pkkanyWC\nBCTYjTHnA88CUcBr1tonAtGuSG1jreVASSk7C0rYVVDM7oISdh0oIX9/MXl7i8jbW8i2PUVs21tE\n3t4itu8rwuuzh7URH+MhtUEdUhok0C2tASkNEkhtWIc2jevSolEdBXgt4HewG2OigBeAc4FcYI4x\n5iNr7TJ/2xapKUp9liJvKYUlPvYXedlX5D30XFBcemjZWVdKQbHz2r5CL7sKSth1oJidBSXsLig5\n1D1yJGOgUWIcjZPiaJIUR/umSTSpF0eTpHgaJ8VxSnICKQ0SaJQYq+Fua7lAHLH3BLKttWsAjDFv\nAxcDCvZaylqLteCzFl/Z86/Lzjpb7jXfkdv7Tmx7b6mPklJLqc9S4vNRWmrx+nx4fRZvqaWk1Ff2\nmqW01FnvbF/R+yzFpT6KSnyHgrrIW0qR1+c8SkoPf/b6KCwpPeqo+XhiozzUiYsiMTaapPhokuvE\n0PqkuiTXiSG5TqzznBBz2HKDOrE0SowlOsoTxH85iRSBCPbmwIZyy7lArwC0e5Tv3nqMxHXT8WGw\nxmBxHj4O/5lD6zwVvm6PsezDgwUsHnxw1DoL+KznsH34jIdSPJQSRal1nr0m6td1ROEtW19qovCW\nrfcShdc623nLlkvL3uesj6KIGEqIppgYiomhyMY4h21A+Rix9telw9dXvk35haq0eVS7HB28tuoZ\nFxaiPIYojyGm7Dk22kNcdBRxMWXP0R7iYzwkJ8QQlxRHfIyz7uDr8eW2i4v2kBgXTd24aBLLHnXj\noqkTG3VoXWy0wlmCKxDBXtHffEf91zbGjAJGAaSlpVVrR/VifNSPKcLYX+P2UCzbIyPbYmwFUX7E\nOuyv8c+h14+//eG/BkLLa2KchyeWUhOL18Ti9cQ4P3tiD73mNXEUexIoiUqg2FOH4qgEij0Jh57L\nry/x1KHYk0BhVF2KoxPBOMFT/h/2yL/sy/+p7zEGj/n12Rjz6zqPwZjDtzHH296U295T+fbGQLTH\nQ3SUOfQc4/E4QR1VFthRznonvD1Eldsm2mPwaPwSiTCBCPZcILXccgqw6ciNrLWvAK8AZGZmVisR\nu179EPBQdd4aPD4f2FLwecs9jrFcWnKM14+xzlsE3kIoLXaevcVEewuJLrdMadGvP5fftmQ3lOyH\ngv1QvN9ZVxXGA/H1IT4ZEpKPfk5oAAkNoe7JULex85zYGKJigvs9i0iVBSLY5wBtjTGtgI3A1cCw\nALRbM3g8gCf8g63UCyUFTsgX74fifeWW90HRPijaAwd2woFdULjLeT6wE3Zv+HWdz1tx+wkNoW4T\n55HYBOo1g/ppkJwK9VOgfqrzi0FEgs7vYLfWeo0xtwNf4Fzu+Ia1dqnflUlgRUVDVD2Ir1f9Nqx1\nfgkU5MO+PNi3FfZvg30HH1thfx5szILlm52/JsqLq+eEfHIaNGwDJ50KjU6FRm0hqenR/T0iUi0B\nuY7dWvsZ8Fkg2pIwZgzEJTmPBi2Pv63P54T87lzYnQO7NpT9vAF2roc134H3wK/bxyZBozbQuB2c\nnA5NO0HTDEhsFNSPJBKJdOepBIfHA0knO4+U7ke/7vPBno2Qvxq2ZzvP+dmw9ntY9M6v2yWd4oR8\nswxI6QkpmVCnYeg+h0gNpGAXd3g8Tv97ciq0GXj4a/u3w5bFsHWJ87xlMWR/Bbbsxp3G7SG1J6T2\nghZ9oGGr0NcvEsYU7BJ+Ek+CNgOcx0FF+2DTPNgwGzbMgWUfwbz/OK81aAmty7Zv1d+5ckekFlOw\nS80QV9cJ7Vb9nWWfz+m+WfMdrJkBi6fA3HHO5ZopPaD9EOgwBBq2drduERcY68JtgpmZmTYrKyvk\n+5UIVloCG+fCLzNg5WewZZGz/uR0J+TTL3NOzIrUYMaYudbazEq3U7BLRNq5HlZ8Css/hpxZgIVT\nukGXYZB+uU7ASo2kYBc5aO9WWDIFFkyCrYvBEwPtzocev3O6dnT9vNQQCnaRimxZDAvfhoWTnBut\nGneAXqMg4yqI1WTMEt4U7CLHU1IIS6fCTy86/fHx9aHHzXDGGOeqHJEwVNVg1/ihUjvFxDv97bd8\nDyO/gFZnwQ9PwzOd4IsHYc9mtysUqTZd7ii1mzGQdobzyFvphPtPL8LPr0DX4dDnLmjQwu0qRU6I\njthFDmrcDi57Ge6Y6xzNz/sPPNcNPrgN8n9xuzqRKlOwixypYSsY+izctRAyb3KuqHk+E977nXNU\nLxLmFOwix1K/Ofzmn3DXIuh9G6z4BF7oBe9e71xdIxKmFOwilUk6GQY/BncvgX6/h+xv4KW+MOka\n2DjP7epEjqJgF6mqxEYw6CG4ZzGc/SdY/yO8OgAmXA45s92uTuQQBbvIiUpoAGc/4BzBD3oYNs2H\nNwbDm0Oc8eRduDdEpDwFu0h1xddzumbuXgyDH4ftq2D8UHjjfMieroAX1yjYRfwVmwhn3u5cRXPB\nk870fxMuh1cHwsppCngJOQW7SKDEJDjjztw537lcsiAfJl0NL/WDpR84Y8iLhICCXSTQouOg+wjn\nRqdLXnQm7Z58A7zYGxZNBl+p2xVKhPMr2I0xTxpjVhhjFhlj3jfGJAeqMJEaLyrGuYP1tp/h8tcB\nA1Nvhud7wPyJzuQgIkHg7xH7V0C6tTYDWAX8yf+SRCKMJwo6XQG3/heufMvpk/9wjDNcQdYb4C1y\nu0KJMH4Fu7X2S2utt2zxJyDF/5JEIpTHAx0vckaUHPYuJDaBT+6BsV1h9stQcsDtCiVCBLKPfSQw\nLYDtiUQmY+C08+Dm6TD8fUhuAdPuh2cy4MexULTP7Qqlhqt0og1jzHSgaQUvPWit/bBsmweBTOAy\ne4wGjTGjgFEAaWlp3devX+9P3SKRZd1M+O6fsPY7SGjojE3Tc5RzrbxImZDNoGSMuQEYDQyy1hZU\n5T2aQUnkGDb8DN8/Cau/dGZ16nUrnDHaudtVar2QzKBkjDkf+CNwUVVDXUSOI7UnXDsZRn0LLfvB\nd0/AvzrB9Edg/3aXi5Oawq8jdmNMNhAH5Jet+slaO7qy9+mIXaSKtiyBH55ybnCKSYDMkXDmHZBU\nUe+oRDpNZi0SSfJWwQ//C4sngycaut/gTNtXXxei1SaazFokkjQ+rWzavizIuNK5/v3ZLvDRnbBz\nndvVSZhRsIvUJA1bw8XPO+PRdLseFk6Csd3g/Vthe7bb1UmYULCL1ETJaTDkaWfavl63wNL34YUe\nMOUm2Lbc7erEZQp2kZqsXjM4/3/g7kXOSdWV0+DfZ8A718HmhW5XJy5RsItEgrpN4NxH4Z4l0P8+\nWPMdvNwf/u8qyJ3rdnUSYgp2kUhSpyEM/Iszq9OAv8CG2fDaQHjrUlg/y+3qJEQU7CKRKCEZzrrP\nCfhz/gZbFsO482HchbDmW83qFOEU7CKRLC4J+t7tnGQ9738gPxv+czG8PhhWf6WAj1AKdpHaILYO\n9B7jzMt64f/C3s0w8Qp4dQCs+FTT9kUYBbtIbRITDz1uhjvmwUXPwYGd8PYweLkfLJmqafsihIJd\npDaKjnVucLp9Llz6sjOL05QbnUslF74Dpd7K25CwpWAXqc2ioqHz1XDbbLhiHHhi4P1R8HwmzHsL\nvMVuVyjVoGAXEWde1vTLYPRMuGqiM8HHR7c787LOeU3zstYwCnYR+ZXHAx2GwKjvYNhkZ3jgT/8A\nz3aGn16EYk27UBMo2EXkaMbAaYPhpq/g+g+hYRv4/AF4NgN+fFbzsoY5BbuIHJsx0PpsuPFTuHEa\nnJwOXz0Ez6TDd09C4W63K5QKKNhFpGpanAnXfwA3fw2pvWDGY860fd88DgU73K5OylGwi8iJScmE\nYe/ALd9D6/7w/T/hmU7w1cOwL8/t6gQFu4hUV7POcNUEuHUWnHae0/f+TCf4/E+wZ7Pb1dVqCnYR\n8c/JHeGKN+D2OXD6pTD7Zecqmk//ALs2uF1drRSQYDfG3GuMscaYkwLRnojUQCe1hUtfhDvmOjc9\nzR0PY7vCR3fAjjVuV1er+B3sxphU4Fwgx/9yRKTGa9gKLhrrzMvafYQzRMFzmTD1Fshb5XZ1tUIg\njtj/BdwPaPxPEflVcipc+JQzbd8Zt8Lyj+CFnjD5Rti61O3qIppfwW6MuQjYaK3V5IoiUrGkpnDe\n486kH33vhtVfwotnwtvXwqYFblcXkYytZKB9Y8x0oGkFLz0I/BkYbK3dbYxZB2Raa7cfo51RwCiA\ntLS07uvXr/enbhGpqQp2wOyX4KeXoGg3tB0M/e+H1B5uVxb2jDFzrbWZlW5XWbAfZwedgK+Bg4NH\npACbgJ7W2i3He29mZqbNysqq1n5FJEIU7oafX4VZL8CBHc4drv3vh5Z93K4sbAU92CvY4TqOc8Re\nnoJdRA4p2gdZb8B/n4P92yDtTGe+1tYDnCEN5JCqBruuYxcRd8XVhT53OidZz/8H7FwHb10Kr50D\nq77QvKzVELBgt9a2rMrRuohIhWIS4IzRcNcCGPIv2LcN/u9KeOUsWP6x5mU9ATpiF5HwEh0HmSPh\nznlw8QtQtBfeuQ5e6gOLp2he1ipQsItIeIqKga7XwW1z4LJXnUB/7yZ4oRcsmKR5WY9DwS4i4S0q\nGjKuhDE/wW/HQ3Q8fDAanu/uDFugeVmPomAXkZrB44HTL4HRP8DVkyChAXx8pzMezc+vQkmh2xWG\nDQW7iNQsxkD738DvZsC170H95vDZvc6IkrNe0LysKNhFpKYyBtqeAyO/gBs+dkaX/OLPzpjwM//l\nnHStpRQJfH/+AAAGCElEQVTsIlKzGQOt+sOIT5yQb9YZpj8C/0qHb/8BB3a5XWHIKdhFJHKknQHD\np8LN3zhztH77d+cI/uv/B/vz3a4uZBTsIhJ5UrrDNZPglh+gzQD44X+dgP/yr86NTxFOwS4ikatZ\nBlz5H+dSyfa/gVnPOwE/7Y+wZ5Pb1QWNgl1EIl+T9nD5a3B7FqRfAXNec66i+eQe2BV5k78p2EWk\n9mjUBi55Ae6YB12uhXlvOdfBf3gb5P/idnUBo2AXkdqnQQsY+gzctRAyb3LGoHk+E977HeStdLs6\nvynYRaT2qt8cfvNPuGsR9L4NVnzijEXz7g2wZYnb1VWbgl1EJOlkGPwY3L0E+v0esr92RpOcNAw2\nznO7uhOmYBcROSixEQx6CO5ZDGf/CdbPhFcHwITLIWe229VVmYJdRORICQ3g7AecI/hBD8Om+fDG\nYBg/FNb+EPazOinYRUSOJb6e0zVz92IY/LhzYnX8EBh3AWRPD9uAV7CLiFQmNhHOvN25iuaCJ51r\n3ydcDq8NgpXTwi7gFewiIlUVkwC9RsGd82Hos7B/O0y6Gl7uB8s+DJt5WRXsIiInKjoOuo+AO+bC\nJS9CyQF493p4sTcsmuz6vKx+B7sx5g5jzEpjzFJjzD8DUZSISI0QFQNdhsFtP8PlrwMGpt4Mz/eA\n+ROhtMSVsvwKdmPMAOBiIMNaezrwVECqEhGpSTxR0OkKuPW/cOVbTp/8h2PguW6QNQ68RaEtx8/3\n3wo8Ya0tArDWRv54mCIix+LxQMeL4JbvYdi7kNgEPrnbGY9m9stOl00oyvDz/acB/Ywxs40x3xlj\negSiKBGRGs0YOO08uHk6DH8fklvAtPudESXXfBf03UdXXp+ZDjSt4KUHy97fADgD6AG8a4xpbe3R\n1/4YY0YBowDS0tL8qVlEpGYwBtoMdB7rZjpzsTY6Nfi7rSCDq/5mYz7H6Yr5tmz5F+AMa23e8d6X\nmZlps7Kyqr1fEZHayBgz11qbWdl2/nbFfAAMLNvhaUAssN3PNkVExA+VdsVU4g3gDWPMEqAYuKGi\nbhgREQkdv4LdWlsMXBegWkREJAB056mISIRRsIuIRBgFu4hIhFGwi4hEGAW7iEiE8esGpWrv1Jg8\nYH3Id+y/k6h91+nrM0e+2vZ5oeZ+5hbW2saVbeRKsNdUxpisqtz1FUn0mSNfbfu8EPmfWV0xIiIR\nRsEuIhJhFOwn5hW3C3CBPnPkq22fFyL8M6uPXUQkwuiIXUQkwijYq8EYc68xxhpjTnK7lmAzxjxp\njFlhjFlkjHnfGJPsdk3BYow5v2xi9mxjzANu1xNsxphUY8wMY8zyssno73K7plAxxkQZY+YbYz5x\nu5ZgULCfIGNMKnAukON2LSHyFZBurc0AVgF/crmeoDDGRAEvABcAHYFrjDEd3a0q6LzAH6y1HXBm\nQbutFnzmg+4ClrtdRLAo2E/cv4D7gVpxcsJa+6W11lu2+BOQ4mY9QdQTyLbWrikbjvpt4GKXawoq\na+1ma+28sp/34gRdc3erCj5jTApwIfCa27UEi4L9BBhjLgI2WmsXul2LS0YC09wuIkiaAxvKLedS\nC0LuIGNMS6ArMNvdSkLiGZyDM5/bhQSLvzMoRZxKJu/+MzA4tBUF3/E+s7X2w7JtHsT5031iKGsL\nIVPBulrxV5kxpi7wHnC3tXaP2/UEkzFmCLDNWjvXGHO22/UEi4L9CNbacypab4zpBLQCFhpjwOmS\nmGeM6Wmt3RLCEgPuWJ/5IGPMDcAQYFAET32YC6SWW04BNrlUS8gYY2JwQn2itXaq2/WEQB/gImPM\nb4B4oJ4xZoK1NqJmgtN17NVkjFkHZFpra+JAQlVmjDkfeBo4y1qb53Y9wWKMicY5OTwI2AjMAYZZ\na5e6WlgQGecIZTyww1p7t9v1hFrZEfu91tohbtcSaOpjl8o8DyQBXxljFhhjXnK7oGAoO0F8O/AF\nzknEdyM51Mv0AYYDA8v+bReUHclKDacjdhGRCKMjdhGRCKNgFxGJMAp2EZEIo2AXEYkwCnYRkQij\nYBcRiTAKdhGRCKNgFxGJMP8fjn0SYvvZRL4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x110fdaf98>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = gluon.loss.RelativeNovelty(rho=3)\n",
    "\n",
    "lossp = loss(output, -nd.ones_like(output))\n",
    "lossq = loss(output, nd.ones_like(output))\n",
    "\n",
    "plt.plot(output.asnumpy(), lossp.asnumpy(), label='Loss for p')\n",
    "plt.plot(output.asnumpy(), lossq.asnumpy(), label='Loss for q')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Exotic Losses\n",
    "\n",
    "There are many loss functions that do not fit into the categories of classification and regression. In fact, there are some recent papers that argue that we should do away with loss functions entirely, such as the one by [Isola, Zhu, Zhou and Efros](https://arxiv.org/abs/1611.07004) from 2016. That said, there are quite a few useful loss functions that are in use. \n",
    "\n",
    "## Triplet Loss\n",
    "\n",
    "Assume that we want to embed data into a vector space. For instance, assume that we want to find embeddings of faces such that faces of the same person are grouped closely together whereas faces of different people are distant. In math: we want $\\|f_a - f_{a'}\\|$ to be small for $a$ and $a'$ drawn from the same class, whereas we want $\\|f_a - f_b\\|$ to be large whenever $a$ and $b$ are from different classes. However, this doesn't really tell us *how small* and *how large* we'd really like these distances to be. There is an easy fix - all we need to do is to enforce that the distances are *relatively large*, namely by some constant $c > 0$.  \n",
    "\n",
    "$$\\|f_a - f_{a'}\\| + c \\leq \\|f_a - f_b\\|$$\n",
    "\n",
    "Now we can use the same trick as for soft-margin losses and turn this into a loss function by taking the maximum over the inequality. One last trick is to square the distances such that gradients look nice and we have the triplet loss:\n",
    "\n",
    "$$l(f_a, f_{a'}, f_b) = \\mathrm{max}(0, c + \\|f_a - f_{a'}\\|^2 - \\|f_a - f_b\\|^2)$$\n",
    "\n",
    "Quite unsurprisingly, this is invoked via the `TripletLoss` class. Its constructor lets us adjust the margin $c$ by which data should be separated. Let's generate some data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "[ 6.16215515  0.          0.          0.          0.        ]\n",
      "<NDArray 5 @cpu(0)>\n",
      "\n",
      "[  0.          36.57266998  51.61498642   9.96375847  32.30460358]\n",
      "<NDArray 5 @cpu(0)>\n"
     ]
    }
   ],
   "source": [
    "loss = gluon.loss.TripletLoss(margin=2)\n",
    "\n",
    "# make some data. f1 and f2 are similar, f3 is (hopefully) far away\n",
    "theshape = (5,3)\n",
    "f1 = nd.normal(shape=theshape)\n",
    "f2 = nd.normal(shape=theshape)\n",
    "f3 = nd.normal(shape=theshape) * 5.0\n",
    "\n",
    "# with the right pair of distances\n",
    "theloss = loss(f1, f2, f3)\n",
    "print(theloss)\n",
    "# these are likely far away in the wrong way, since we blew f3 out of proportions\n",
    "theloss = loss(f1, f3, f2)\n",
    "print(theloss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Todo - many hierarchical losses, pairwise loss with margin, minimum error decoder, etc."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
